Bases: exceptions.Exception
Bases: exceptions.Exception
Bases: eventlet.timeout.Timeout
Bases: eventlet.timeout.Timeout
Bases: eventlet.timeout.Timeout
Bases: exceptions.Exception
Bases: exceptions.Exception
Bases: exceptions.Exception
Bases: eventlet.timeout.Timeout
Max container list length of a get request for an account
Max object list length of a get request for a container
Max file size allowed for objects
Max number of metadata items
Max length of the name of a key for metadata
Max overall size of metadata
Max length of the value of a key for metadata
Max object name length
Helper function for checking if a string can be converted to a float.
Parameters: |
|
---|---|
Returns: | True if the string can be converted to a float, False otherwise |
Check metadata sent in the request headers.
Parameters: |
|
---|---|
Raises HTTPBadRequest: | |
bad metadata |
Verify that the path to the device is a mount point and mounted. This allows us to fast fail on drives that have been unmounted because of issues, and also prevents us for accidently filling up the root partition.
Parameters: |
|
---|---|
Returns: | True if it is a valid mounted device, False otherwise |
Check to ensure that everything is alright about an object to be created.
Parameters: |
|
---|---|
Raises: |
|
Validate if a string is valid UTF-8.
Parameters: |
|
---|---|
Returns: | True if the string is valid utf-8, False otherwise |
Miscellaneous utility functions for use with Swift.
Bases: eventlet.greenpool.GreenPool
GreenPool subclassed to kill its coros when it gets gc’ed
Bases: logging.LoggerAdapter, object
A Logger like object which performs some reformatting on calls to exception(). Can be used to store a threadlocal transaction id.
Convenience function for syslog priority LOG_NOTICE. The python logging lvl is set to 25, just above info. SysLogHandler is monkey patched to map this log lvl to the LOG_NOTICE syslog priority.
Add extra info to message
Bases: urlparse.ParseResult
Parse results class for urlparse.
A no-op logger for eventlet wsgi.
Bases: logging.Formatter
Custom logging.Formatter will append txn_id to a log message if the record has one and the message does not.
Given a devices path and a data directory, yield (path, device, partition) for all files in that directory
Parameters: |
|
---|
Get memcache connection pool from the environment (which had been previously set by the memcache middleware
Parameters: |
|
---|---|
Returns: | swift.common.memcached.MemcacheRing from environment |
Log unhandled exceptions, close stdio, capture stdout and stderr.
param logger: Logger object to use
Compute an ETA. Now only if we could also have a progress bar...
Parameters: |
|
---|---|
Returns: | ETA as a tuple of (length of time, unit of time) where unit of time is one of (‘h’, ‘m’, ‘s’) |
Drop ‘buffer’ cache for the given range of the given file.
Parameters: |
|
---|
Sets the userid/groupid of the current process, get session leader, etc.
Parameters: |
|
---|
Pre-allocate disk space for a file file.
Parameters: |
|
---|
Get the current system logger using config settings.
Log config and defaults:
log_facility = LOG_LOCAL0
log_level = INFO
log_name = swift
Parameters: |
|
---|
Get parameters from an HTTP request ensuring proper handling UTF-8 encoding.
Parameters: |
|
---|---|
Returns: | HTTP request parameter value |
Get a nomralized length of time in the largest unit of time (hours, minutes, or seconds.)
Parameters: |
|
---|---|
Returns: | A touple of (length of time, unit of time) where unit of time is one of (‘h’, ‘m’, ‘s’) |
Get the connonical hash for an account/container/object
Parameters: |
|
---|---|
Returns: | hash string |
Returns the number in a human readable format; for example 1048576 = “1Mi”.
Get a value from the wsgi environment
Parameters: |
|
---|---|
Returns: | the value from the environment |
Iterate over partitions accross all devices.
Parameters: |
|
---|---|
Returns: | Each iteration returns a tuple of (device, partition) |
Attempt to find the function in libc, otherwise return a no-op func.
Parameters: |
|
---|
Context manager that acquires a lock on the parent directory of the given file path. This will block until the lock can be acquired, or the timeout time has expired (whichever occurs first).
Parameters: |
|
---|
Context manager that acquires a lock on a directory. This will block until the lock can be acquired, or the timeout time has expired (whichever occurs first).
Parameters: |
|
---|
Ensures the path is a directory or makes it if not. Errors if the path exists but is a file or on permissions failure.
Parameters: |
|
---|
Format a timestamp (string or numeric) into a standardized xxxxxxxxxx.xxxxx format.
Parameters: |
|
---|---|
Returns: | normalized timestamp as a string |
Parse standard swift server/daemon options with optparse.OptionParser.
Parameters: |
|
---|
Raises SystemExit: | |
---|---|
First arg (CONFIG) is required, file must exist |
Will eventlet.sleep() for the appropriate time so that the max_rate is never exceeded. If max_rate is 0, will not ratelimit. The maximum recommended rate should not exceed (1000 * incr_by) a second as eventlet.sleep() does involve some overhead. Returns running_time that should be used for subsequent calls.
Parameters: |
|
---|
Read config file and return config items as a dict
Parameters: |
|
---|---|
Returns: | dict of config items |
Quiet wrapper for os.unlink, OSErrors are suppressed
Parameters: |
|
---|
Attempt to fix / hide race conditions like empty object directories being removed by backend processes during uploads, by retrying.
Parameters: |
|
---|
Look in root, for any files/dirs matching glob, recurively traversing any found directories looking for files ending with ext
Parameters: |
|
---|---|
Returns: | list of full paths to matching files, sorted |
Validate and split the given HTTP request path.
Examples:
['a'] = split_path('/a')
['a', None] = split_path('/a', 1, 2)
['a', 'c'] = split_path('/a/c', 1, 2)
['a', 'c', 'o/r'] = split_path('/a/c/o/r', 1, 3, True)
Parameters: |
|
---|---|
Returns: | list of segments with a length of maxsegs (non-existant segments will return as None) |
Raises : | ValueError if given an invalid path |
Get the storage directory
Parameters: |
|
---|---|
Returns: | Storage directory |
Remove any file in a given path that that was last modified before mtime.
Parameters: |
|
---|---|
Mtime : | timestamp of oldest file to keep |
urlparse augmentation. This is necessary because urlparse can’t handle RFC 2732 URLs.
Parameters: |
|
---|
Get the machine’s ip addresses
Returns: | list of Strings of ip addresses |
---|
Write contents to file at path
Parameters: |
|
---|
Ensure that a pickle file gets written to disk. The file is first written to a tmp location, ensure it is synced to disk, then perform a move to its final location
Parameters: |
|
---|
Bases: object
Scalable authentication and authorization system that uses Swift as its backing store.
Parameters: |
|
---|
Returns None if the request is authorized to continue or a standard WSGI response callable if not.
Returns True if the key is valid for the user_detail. Currently, this only supports plaintext key matching.
Parameters: |
|
---|---|
Returns: | True if the key is valid for the user, False if not. |
Returns a standard WSGI response callable with the status of 403 or 401 depending on whether the REMOTE_USER is set or not.
Returns the dict for the user specified as the admin in the request with the addition of an account key set to the admin user’s account.
Parameters: |
|
---|---|
Returns: | The dict for the admin user with the addition of the account key. |
Returns an HTTPConnection based on the urlparse result given or the default Swift cluster (internal url) urlparse result.
Parameters: |
|
---|
Get groups for the given token.
Parameters: |
|
---|---|
Returns: | None if the token is invalid or a string containing a comma separated list of groups the authenticated user is a member of. The first group in the list is also considered a unique identifier for that user. |
Returns the current internal token to use for the auth system’s own actions with other services. Each process will create its own itoken and the token will be deleted and recreated based on the token_life configuration value. The itoken information is stored in memcache because the auth process that is asked by Swift to validate the token may not be the same as the auth process that created the token.
WSGI entry point for auth requests (ones that match the self.auth_prefix). Wraps env in webob.Request object and passes it down.
Parameters: |
|
---|
Handles the DELETE v2/<account> call for removing an account from the auth system. Can only be called by a .reseller_admin.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success. |
Handles the DELETE v2/<account>/<user> call for deleting a user from an account.
Can only be called by an account .admin.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success. |
Handles the GET v2/<account> call for getting account information. Can only be called by an account .admin.
On success, a JSON dictionary will be returned containing the keys account_id, services, and users. The account_id is the value used when creating service accounts. The services value is a dict as described in the handle_get_token() call. The users value is a list of dicts, each dict representing a user and currently only containing the single key name. For example:
{"account_id": "AUTH_018c3946-23f8-4efb-a8fb-b67aae8e4162",
"services": {"storage": {"default": "local",
"local": "http://127.0.0.1:8080/v1/AUTH_018c3946-23f8-4efb-a8fb-b67aae8e4162"}},
"users": [{"name": "tester"}, {"name": "tester3"}]}
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with a JSON dictionary as explained above. |
Handles the GET v2 call for getting general reseller information (currently just a list of accounts). Can only be called by a .reseller_admin.
On success, a JSON dictionary will be returned with a single accounts key whose value is list of dicts. Each dict represents an account and currently only contains the single key name. For example:
{"accounts": [{"name": "reseller"}, {"name": "test"},
{"name": "test2"}]}
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with a JSON dictionary as explained above. |
Handles the various request for token and service end point(s) calls. There are various formats to support the various auth servers in the past. Examples:
GET <auth-prefix>/v1/<act>/auth
X-Auth-User: <act>:<usr> or X-Storage-User: <usr>
X-Auth-Key: <key> or X-Storage-Pass: <key>
GET <auth-prefix>/auth
X-Auth-User: <act>:<usr> or X-Storage-User: <act>:<usr>
X-Auth-Key: <key> or X-Storage-Pass: <key>
GET <auth-prefix>/v1.0
X-Auth-User: <act>:<usr> or X-Storage-User: <act>:<usr>
X-Auth-Key: <key> or X-Storage-Pass: <key>
On successful authentication, the response will have X-Auth-Token and X-Storage-Token set to the token to use with Swift and X-Storage-URL set to the URL to the default Swift cluster to use.
The response body will be set to the account’s services JSON object as described here:
{"storage": { # Represents the Swift storage service end points
"default": "cluster1", # Indicates which cluster is the default
"cluster1": "<URL to use with Swift>",
# A Swift cluster that can be used with this account,
# "cluster1" is the name of the cluster which is usually a
# location indicator (like "dfw" for a datacenter region).
"cluster2": "<URL to use with Swift>"
# Another Swift cluster that can be used with this account,
# there will always be at least one Swift cluster to use or
# this whole "storage" dict won't be included at all.
},
"servers": { # Represents the Nova server service end points
# Expected to be similar to the "storage" dict, but not
# implemented yet.
},
# Possibly other service dicts, not implemented yet.
}
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with data set as explained above. |
Handles the GET v2/<account>/<user> call for getting user information. Can only be called by an account .admin.
On success, a JSON dict will be returned as described:
{"groups": [ # List of groups the user is a member of
{"name": "<act>:<usr>"},
# The first group is a unique user identifier
{"name": "<account>"},
# The second group is the auth account name
{"name": "<additional-group>"}
# There may be additional groups, .admin being a special
# group indicating an account admin and .reseller_admin
# indicating a reseller admin.
],
"auth": "plaintext:<key>"
# The auth-type and key for the user; currently only plaintext is
# implemented.
}
For example:
{"groups": [{"name": "test:tester"}, {"name": "test"},
{"name": ".admin"}],
"auth": "plaintext:testing"}
If the <user> in the request is the special user .groups, the JSON dict will contain a single key of groups whose value is a list of dicts representing the active groups within the account. Each dict currently has the single key name. For example:
{"groups": [{"name": ".admin"}, {"name": "test"},
{"name": "test:tester"}, {"name": "test:tester3"}]}
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with a JSON dictionary as explained above. |
Handles the POST v2/.prep call for preparing the backing store Swift cluster for use with the auth subsystem. Can only be called by .super_admin.
Parameters: |
|
---|---|
Returns: | webob.Response, 204 on success |
Handles the PUT v2/<account> call for adding an account to the auth system. Can only be called by a .reseller_admin.
By default, a newly created UUID4 will be used with the reseller prefix as the account id used when creating corresponding service accounts. However, you can provide an X-Account-Suffix header to replace the UUID4 part.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success. |
Handles the PUT v2/<account>/<user> call for adding a user to an account.
X-Auth-User-Key represents the user’s key, X-Auth-User-Admin may be set to true to create an account .admin, and X-Auth-User-Reseller-Admin may be set to true to create a .reseller_admin.
Can only be called by an account .admin unless the user is to be a .reseller_admin, in which case the request must be by .super_admin.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success. |
Entry point for auth requests (ones that match the self.auth_prefix). Should return a WSGI-style callable (such as webob.Response).
Parameters: |
|
---|
Handles the POST v2/<account>/.services call for setting services information. Can only be called by a reseller .admin.
In the handle_get_account() (GET v2/<account>) call, a section of the returned JSON dict is services. This section looks something like this:
"services": {"storage": {"default": "local",
"local": "http://127.0.0.1:8080/v1/AUTH_018c3946-23f8-4efb-a8fb-b67aae8e4162"}}
Making use of this section is described in handle_get_token().
This function allows setting values within this section for the <account>, allowing the addition of new service end points or updating existing ones.
The body of the POST request should contain a JSON dict with the following format:
{"service_name": {"end_point_name": "end_point_value"}}
There can be multiple services and multiple end points in the same call.
Any new services or end points will be added to the existing set of services and end points. Any existing services with the same service name will be merged with the new end points. Any existing end points with the same end point name will have their values updated.
The updated services dictionary will be returned on success.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with the udpated services JSON dict as described above |
Handles the GET v2/.token/<token> call for validating a token, usually called by a service like Swift.
On a successful validation, X-Auth-TTL will be set for how much longer this token is valid and X-Auth-Groups will contain a comma separated list of groups the user belongs to.
The first group listed will be a unique identifier for the user the token represents.
.reseller_admin is a special group that indicates the user should be allowed to do anything on any account.
Parameters: |
|
---|---|
Returns: | webob.Response, 2xx on success with data set as explained above. |
Returns True if the admin specified in the request represents a .admin for the account specified.
Parameters: |
|
---|
Returns True if the admin specified in the request represents a .reseller_admin.
Parameters: |
|
---|
Returns True if the admin specified in the request represents the .super_admin.
Parameters: |
|
---|
Makes a new webob.Request based on the current env but with the parameters specified.
Parameters: |
|
---|---|
Returns: | webob.Request object |
Returns a WSGI filter app for use with paste.deploy.
Returns a cleaned ACL header value, validating that it meets the formatting requirements for standard Swift ACL strings.
The ACL format is:
[item[,item...]]
Each item can be a group name to give access to or a referrer designation to grant or deny based on the HTTP Referer header.
The referrer designation format is:
.r:[-]value
The .r can also be .ref, .referer, or .referrer; though it will be shortened to just .r for decreased character count usage.
The value can be * to specify any referrer host is allowed access, a specific host name like www.example.com, or if it has a leading period . or leading *. it is a domain name specification, like .example.com or *.example.com. The leading minus sign - indicates referrer hosts that should be denied access.
Referrer access is applied in the order they are specified. For example, .r:.example.com,.r:-thief.example.com would allow all hosts ending with .example.com except for the specific host thief.example.com.
Example valid ACLs:
.r:*
.r:*,.r:-.thief.com
.r:*,.r:.example.com,.r:-thief.example.com
.r:*,.r:-.thief.com,bobs_account,sues_account:sue
bobs_account,sues_account:sue
Example invalid ACLs:
.r:
.r:-
By default, allowing read access via .r will not allow listing objects in the container – just retrieving objects from the container. To turn on listings, use the .rlistings directive.
Also, .r designations aren’t allowed in headers whose names include the word ‘write’.
ACLs that are “messy” will be cleaned up. Examples:
Original | Cleaned |
bob, sue | bob,sue |
bob , sue | bob,sue |
bob,,,sue | bob,sue |
.referrer : * | .r:* |
.ref:*.example.com | .r:.example.com |
.r:*, .rlistings | .r:*,.rlistings |
Parameters: |
|
---|---|
Returns: | The value, cleaned of extraneous formatting. |
Raises ValueError: | |
If the value does not meet the ACL formatting requirements; the error message will indicate why. |
Parses a standard Swift ACL string into a referrers list and groups list.
See clean_acl() for documentation of the standard Swift ACL format.
Parameters: |
|
---|---|
Returns: | A tuple of (referrers, groups) where referrers is a list of referrer designations (without the leading .r:) and groups is a list of groups to allow access. |
Returns True if the referrer should be allowed based on the referrer_acl list (as returned by parse_acl()).
See clean_acl() for documentation of the standard Swift ACL format.
Parameters: |
|
---|---|
Returns: | True if the referrer should be allowed; False if not. |
WSGI tools for use with swift.
Bind socket to bind ip:port in conf
Parameters: |
|
---|
mimetools.Message defaults content-type to “text/plain” This changes it to default to None, so we can detect missing headers.
Loads common settings from conf, then instantiates app and runs the server using the specified number of workers.
Parameters: |
|
---|
Cloud Files client library used internally
Bases: exceptions.Exception
Bases: object
Convenience class to make requests that will also retry the request
Wrapper for delete_container()
Wrapper for delete_object()
Wrapper for get_account()
Wrapper for get_container()
Wrapper for get_object()
Wrapper for head_account()
Wrapper for head_container()
Wrapper for head_object()
Wrapper for post_account()
Wrapper for post_container()
Wrapper for post_object()
Wrapper for put_container()
Wrapper for put_object()
Delete a container
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP DELETE request failed |
Delete object
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP DELETE request failed |
Get a listing of containers for the account.
Parameters: |
|
---|---|
Returns: | a tuple of (response headers, a list of containers) The response headers will be a dict and all header names will be lowercase. |
Raises ClientException: | |
HTTP GET request failed |
Get authentication/authorization credentials.
The snet parameter is used for Rackspace’s ServiceNet internal network implementation. In this function, it simply adds snet- to the beginning of the host name for the returned storage URL. With Rackspace Cloud Files, use of this network path causes no bandwidth charges but requires the client to be running on Rackspace’s ServiceNet network.
Parameters: |
|
---|---|
Returns: | tuple of (storage URL, auth token) |
Raises ClientException: | |
HTTP GET request to auth URL failed |
Get a listing of objects for the container.
Parameters: |
|
---|---|
Returns: | a tuple of (response headers, a list of objects) The response headers will be a dict and all header names will be lowercase. |
Raises ClientException: | |
HTTP GET request failed |
Get an object
Parameters: |
|
---|---|
Returns: | a tuple of (response headers, the object’s contents) The response headers will be a dict and all header names will be lowercase. |
Raises ClientException: | |
HTTP GET request failed |
Get account stats.
Parameters: |
|
---|---|
Returns: | a dict containing the response’s headers (all header names will be lowercase) |
Raises ClientException: | |
HTTP HEAD request failed |
Get container stats.
Parameters: |
|
---|---|
Returns: | a dict containing the response’s headers (all header names will be lowercase) |
Raises ClientException: | |
HTTP HEAD request failed |
Get object info
Parameters: |
|
---|---|
Returns: | a dict containing the response’s headers (all header names will be lowercase) |
Raises ClientException: | |
HTTP HEAD request failed |
Make an HTTPConnection or HTTPSConnection
Parameters: |
|
---|---|
Returns: | tuple of (parsed url, connection object) |
Raises ClientException: | |
Unable to handle protocol scheme |
Update an account’s metadata.
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP POST request failed |
Update a container’s metadata.
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP POST request failed |
Update object metadata
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP POST request failed |
Create a container
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP PUT request failed |
Put an object
Parameters: |
|
---|---|
Returns: | etag from server response |
Raises ClientException: | |
HTTP PUT request failed |
Patched version of urllib.quote that encodes utf8 strings before quoting
Internal client library for making calls directly to the servers rather than through the proxy.
Delete object directly from the object server.
Parameters: |
|
---|---|
Returns: | response from server |
Get container listings directly from the container server.
Parameters: |
|
---|---|
Returns: | a tuple of (response headers, a list of objects) The response headers will be a dict and all header names will be lowercase. |
Get object directly from the object server.
Parameters: |
|
---|---|
Returns: | a tuple of (response headers, the object’s contents) The response headers will be a dict and all header names will be lowercase. |
Request container information directly from the container server.
Parameters: |
|
---|---|
Returns: | a dict containing the response’s headers (all header names will be lowercase) |
Request object information directly from the object server.
Parameters: |
|
---|---|
Returns: | a dict containing the response’s headers (all header names will be lowercase) |
Direct update to object metadata on object server.
Parameters: |
|
---|---|
Raises ClientException: | |
HTTP POST request failed |
Put object directly from the object server.
Parameters: |
|
---|---|
Returns: | etag from the server response |
Helper function to retry a given function a number of times.
Parameters: |
|
---|---|
Returns: | restult of func |
Monkey Patch httplib.HTTPResponse to buffer reads of headers. This can improve performance when making large numbers of small HTTP requests. This module also provides helper functions to make HTTP connections using BufferedHTTPResponse.
Warning
If you use this, be sure that the libraries you are using do not access the socket directly (xmlrpclib, I’m looking at you :/), and instead make all calls through httplib.
Bases: httplib.HTTPConnection
HTTPConnection class that uses BufferedHTTPResponse
alias of BufferedHTTPResponse
Bases: httplib.HTTPResponse
HTTPResponse class that buffers reading of headers
Helper function to create an HTTPConnection object. If ssl is set True, HTTPSConnection will be used. However, if ssl=False, BufferedHTTPConnection will be used, which is buffered for backend Swift services.
Parameters: |
|
---|---|
Returns: | HTTPConnection object |
Helper function to create an HTTPConnection object. If ssl is set True, HTTPSConnection will be used. However, if ssl=False, BufferedHTTPConnection will be used, which is buffered for backend Swift services.
Parameters: |
|
---|---|
Returns: | HTTPConnection object |
Lucid comes with memcached: v1.4.2. Protocol documentation for that version is at:
http://github.com/memcached/memcached/blob/1.4.2/doc/protocol.txt
Bases: object
Simple, consistent-hashed memcache client.
Decrements a key which has a numeric value by delta. Calls incr with -delta.
Parameters: |
|
---|---|
Raises MemcacheConnectionError: | |
Deletes a key/value pair from memcache.
Parameters: |
|
---|
Gets the object specified by key. It will also unpickle the object before returning if it is pickled in memcache.
Parameters: |
|
---|---|
Returns: | value of the key in memcache |
Gets multiple values from memcache for the given keys.
Parameters: |
|
---|---|
Returns: | list of values |
Increments a key which has a numeric value by delta. If the key can’t be found, it’s added as delta or 0 if delta < 0. If passed a negative number, will use memcached’s decr. Returns the int stored in memcached Note: The data memcached stores as the result of incr/decr is an unsigned int. decr’s that result in a number below 0 are stored as 0.
Parameters: |
|
---|---|
Raises MemcacheConnectionError: | |
Set a key/value pair in memcache
Parameters: |
|
---|
Sets multiple key/value pairs in memcache.
Parameters: |
|
---|
Main class for performing commands on groups of servers.
Parameters: |
|
---|
alias for reload
Find and return the decorated method named like cmd
Parameters: |
|
---|
Get all publicly accessible commands
Returns: | a list of string tuples (cmd, help), the method names who are decorated as commands |
---|
start a server interactively
spawn server and return immediately
start server and run one pass on supporting daemons
graceful shutdown then restart on supporting servers
stops then restarts server
Find the named command and run it
Parameters: |
|
---|
allow current requests to finish on supporting servers
starts a server
display status of tracked pids for server
stops a server
Manage operations on a server or group of servers of similar type
Parameters: |
|
---|
Get conf files for this server
Param : | number, if supplied will only lookup the nth server |
---|---|
Returns: | list of conf files |
Translate pid_file to a corresponding conf_file
Parameters: |
|
---|---|
Returns: | the conf_file for this pid_file |
Translate conf_file to a corresponding pid_file
Parameters: |
|
---|---|
Returns: | the pid_file for this conf_file |
Get running pids
Returns: | a dict mapping pids (ints) to pid_files (paths) |
---|
wait on spawned procs to terminate
Generator, yields (pid_file, pids)
Kill running pids
Parameters: |
|
---|---|
Returns: | a dict mapping pids (ints) to pid_files (paths) |
Collect conf files and attempt to spawn the processes for this server
Get pid files for this server
Param : | number, if supplied will only lookup the nth server |
---|---|
Returns: | list of pid files |
Send a signal to pids for this server
Parameters: |
|
---|---|
Returns: | a dict mapping pids (ints) to pid_files (paths) |
Launch a subprocess for this server.
Parameters: |
|
---|
:returns : the pid of the spawned process
Display status of server
Param : | pids, if not supplied pids will be populated automatically |
---|---|
Param : | number, if supplied will only lookup the nth server |
Returns: | 1 if server is not running, 0 otherwise |
Send stop signals to pids for this server
Returns: | a dict mapping pids (ints) to pid_files (paths) |
---|
wait on spawned procs to start
Decorator to declare which methods are accessible as commands, commands always return 1 or 0, where 0 should indicate success.
Parameters: |
|
---|
Try to increase resource limits of the OS. Move PYTHON_EGG_CACHE to /tmp
Monitor a collection of server pids yeilding back those pids that aren’t responding to signals.
Parameters: |
|
---|
Bases: object
Rate limiting middleware
Rate limits requests on both an Account and Container level. Limits are configurable.
Returns number of requests allowed per second for given container size.
Returns a list of key (used in memcache), ratelimit tuples. Keys should be checked in order.
Parameters: |
|
---|
Performs rate limiting and account white/black listing. Sleeps if necessary.
Parameters: |
|
---|
paste.deploy app factory for creating WSGI proxy apps.
The swift3 middleware will emulate the S3 REST api on top of swift.
The boto python library is necessary to use this middleware (install the python-boto package if you use Ubuntu).
The following opperations are currently supported:
- GET Service
- DELETE Bucket
- GET Bucket (List Objects)
- PUT Bucket
- DELETE Object
- GET Object
- HEAD Object
- PUT Object
- PUT Object (Copy)
To add this middleware to your configuration, add the swift3 middleware in front of the auth middleware, and before any other middleware that look at swift requests (like rate limiting).
To set up your client, the access key will be the concatenation of the account and user strings that should look like test:tester, and the secret access key is the account password. The host should also point to the swift storage hostname. It also will have to use the old style calling format, and not the hostname based container format.
An example client using the python boto library might look like the following for an SAIO setup:
connection = boto.s3.Connection(
aws_access_key_id='test:tester',
aws_secret_access_key='testing',
port=8080,
host='127.0.0.1',
is_secure=False,
calling_format=boto.s3.connection.OrdinaryCallingFormat())
Bases: swift.common.middleware.swift3.Controller
Handles bucket request.
Handle DELETE Bucket request
Handle GET Bucket (List Objects) request
Handle PUT Bucket request
Bases: swift.common.middleware.swift3.Controller
Handles requests on objects
Handle DELETE Object request
Handle GET Object request
Handle HEAD Object request
Handle PUT Object and PUT Object (Copy) request
Bases: swift.common.middleware.swift3.Controller
Handles account level requests.
Handle GET Service request
Bases: object
Swift3 S3 compatibility midleware
Canonicalize a request to a token that can be signed.
Standard filter factory to use the middleware with paste.deploy
Given an HTTP response code, create a properly formatted xml error response
Parameters: |
|
---|---|
Returns: | webob.response object |
This StaticWeb WSGI middleware will serve container data as a static web site with index file and error file resolution and optional file listings. This mode is normally only active for anonymous requests. If you want to use it with authenticated requests, set the X-Web-Mode: true header on the request.
The staticweb filter should be added to the pipeline in your /etc/swift/proxy-server.conf file just after any auth middleware. Also, the configuration section for the staticweb middleware itself needs to be added. For example:
[DEFAULT]
...
[pipeline:main]
pipeline = healthcheck cache swauth staticweb proxy-server
...
[filter:staticweb]
use = egg:swift#staticweb
# Seconds to cache container x-container-meta-web-* header values.
# cache_timeout = 300
# You can override the default log routing for this filter here:
# set log_name = staticweb
# set log_facility = LOG_LOCAL0
# set log_level = INFO
# set access_log_name = staticweb
# set access_log_facility = LOG_LOCAL0
# set access_log_level = INFO
# set log_headers = False
Any publicly readable containers (for example, X-Container-Read: .r:*, see acls for more information on this) will be checked for X-Container-Meta-Web-Index and X-Container-Meta-Web-Error header values:
X-Container-Meta-Web-Index <index.name>
X-Container-Meta-Web-Error <error.name.suffix>
If X-Container-Meta-Web-Index is set, any <index.name> files will be served without having to specify the <index.name> part. For instance, setting X-Container-Meta-Web-Index: index.html will be able to serve the object .../pseudo/path/index.html with just .../pseudo/path or .../pseudo/path/
If X-Container-Meta-Web-Error is set, any errors (currently just 401 Unauthorized and 404 Not Found) will instead serve the .../<status.code><error.name.suffix> object. For instance, setting X-Container-Meta-Web-Error: error.html will serve .../404error.html for requests for paths not found.
For psuedo paths that have no <index.name>, this middleware can serve HTML file listings if you set the X-Container-Meta-Web-Listings: true metadata item on the container.
If listings are enabled, the listings can have a custom style sheet by setting the X-Container-Meta-Web-Listings-CSS header. For instance, setting X-Container-Meta-Web-Listings-CSS: listing.css will make listings link to the .../listing.css style sheet. If you “view source” in your browser on a listing page, you will see the well defined document structure that can be styled.
Example usage of this middleware via st:
Make the container publicly readable:
st post -r '.r:*' containerYou should be able to get objects directly, but no index.html resolution or listings.
Set an index file directive:
st post -m 'web-index:index.html' containerYou should be able to hit paths that have an index.html without needing to type the index.html part.
Turn on listings:
st post -m 'web-listings: true' containerNow you should see object listings for paths and pseudo paths that have no index.html.
Enable a custom listings style sheet:
st post -m 'web-listings-css:listings.css' containerSet an error file:
st post -m 'web-error:error.html' containerNow 401’s should load 401error.html, 404’s should load 404error.html, etc.
Bases: object
The Static Web WSGI middleware filter; serves container data as a static web site. See staticweb for an overview.
Parameters: |
|
---|
Web access logger for this filter.
The next WSGI application/filter in the paste.deploy pipeline.
The seconds to cache the x-container-meta-web-* headers.,
The filter configuration dict.
Indicates whether full HTTP headers should be logged or not.
Logger for this filter.
Returns a Static Web WSGI filter for use with paste.deploy.