OpenStack Swift is great for large amounts of data that comes from–and is used by–many, many clients. Think about uploading photos from phones or playing online games or downloading software updates. These use cases require instant access to data and massive concurrency. Swift is tailor-made for these use cases, but there’s often one small detail that makes multi-tenant object storage hard: auth.
If you don’t want to give wide-open public access to you data, you must be able to authorize a particular request for the data. The simple answer is to create a new user for every person (or program!) that could access the data, and require those users to send an auth token with the request. Unfortunately, this method doesn’t scale well. If you have hundreds, thousands, or millions of users, you do not want to mange the overhead of an auth identity for each user, especially if all those users are doing is a single read or write operation (e.g. downloading a software update or uploading a photo).
The solution is to use signed URLs to access the data, and Swift natively supports these kind of requests. Signed requests, or “temp-urls” as they are called in Swift, allow you to create a URL that is good for a specific action for a specific amount of time. It’s a perfect way to allow your users to directly upload and download data without requiring that they create a user identity or user non-standard HTTP headers in the request.
TempURLs are Great!
Here’s lighthearted take on why you should use TempURLs with Swift. It’s a rerecording of a lightning talk I gave at the OpenStack summit in Paris last November.
TempURLs are automatically enabled and supported in every SwiftStack installation. To get started, take advantage of our free test/dev program
First, choose a secret and set it on the account metadata:
swift post -m "Temp-URL-Key: my_super_secret_key"
Using curl, the equvalent command would be:
curl -i -H "X-Auth-Token: $TOKEN" -H "X-Account-Meta-Temp-URL-Key: my_super_secret_key" $STORAGEURL -XPOST
Note that in both of the above examples, I’ve glossed over the auth credentials. See auth usage docs for more info.
You can set up to two keys on an account. This allows you to establish a key rotation scheme so that you can securely rotate your keys without worrying about invalidating existing TempURLs that may still be in use by clients.
For example, if you always create TempURLs with an expiry time of less than one hour (3600 seconds), then you know that you can safely change one key every hour without breaking existing links.
The valid TempURL keys are
One of the great advantages of TempURLs is that they can be created very easily without needing to do any network or other remote resource access. If you have the shared secret, you can create new, valid TempURLs all day long
Choose a name for the object you are uploading. This might be user-selected or auto-generated based on some method in your app. For this example, I’ll use the name “catphoto.jpg”.
Next, using one of the TempURL keys set on the account (see above), create a TempURL:
swift tempurl PUT 3600 /v1/AUTH_myaccount/catpictures/catphoto.jpg my_super_secret_key
Now, for the next hour (3600 seconds) anyone with the resulting URL will be able to upload data to the object called catphoto.jpg
If you want to create a read-only link, you’d create the TempURL to allow the GET verb:
swift tempurl GET 3600 /v1/AUTH_myaccount/catpictures/catphoto.jpg my_super_secret_key
This generates a time-limited URL path (e.g.
/v1/AUTH_myaccount/catpictures/catphoto.jpg?temp_url_sig=d17b08a491b8b784bb4542bb4cee73ec3c3404c7&temp_url_expires=1422501571), and anyone can use it to read the object named catphoto.jpg for the next hour.
By default a Content-Disposition header will be set on the response so that browsers will interpret this as a file attachment to be saved. The filename chosen is based on the object name, but you can override this with a filename query parameter.
To set your own download filename, add a “filename” query parameter to the generated TempURL, eg
If you do not want to have the Content-Disposition header set, add an “inline” query parameter to the generate TempURL:
On a cluster-wide basis, incoming (request) and outgoing (response) headers can be allowed or denied. For example, this can be used to allow someone to upload an object but not allow them to set particular metadata about the object. Or it could be used to prevent certain metadata from being exposed without normal authentication and authorization. Seehttps://swiftstack.com/docs/admin/middleware/tempurl.html for information on how to configure this in the SwiftStack controller.
TempURLs are an HMAC-SHA1 signature of the request being made using the key set on the account metadata and the HTTP method, the expiry time (in seconds), and the full path to the object. From the documentation, here’s a small code snippet to generate the signature for a GET for 60 seconds on /v1/AUTH_account/container/object:
import hmac from hashlib import sha1 from time import time method = 'GET' expires = int(time() + 60) path = '/v1/AUTH_account/container/object' key = 'mykey' hmac_body = '%s\\n%s\\n%s' % (method, expires, path) sig = hmac.new(key, hmac_body, sha1).hexdigest()
Be certain to use the full path, from the /v1/ onward.
Assuming the sig ends up equaling da39a3ee5e6b4b0d3255bfef95601890afd80709 and expires ends up as 1323479485. Then a link to:
https://swift-cluster.example.com/v1/AUTH_account/container/object? temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709& temp_url_expires=1323479485
would grant GET (and HEAD) access until the expiry time. Any alteration of the request would result in a 401 response code. Note that HEAD is allowed if GET, PUT, or POST is allowed.
Go use them!
Go use TempURLs and let us know how you are using them! We love to hear your stories, and we’re here to answer any questions you have.
If you want to get started with Swift, check out our free trial. You’ll be up and running with OpenStack Swift in minutes.