On July 1, the OpenStack Swift community completed the at-rest data encryption feature for Swift. I’d like to go over what this feature does, what it protects against, how it works, and where it should be used.
At-rest encryption in Swift encrypts all object data, object etags, and any user metadata values set on objects. The feature is enabled by a cluster operator and completely transparent to the end-user. Internally-encrypted data is never returned to clients via the API.
All security features start with defining the threat model, ie the circumstances you’re trying to protect against. For the at-rest encryption feature in Swift, the threat model is rather straightforward: protect object data and user metadata from being exposed if a data drive leaves the Swift cluster. There are two common cases where data drives might leave a Swift cluster. The first is by accident: an inventory error could misplace a drive taken out of a Swift cluster and put it into a different server. If the drive isn’t erased, the data would be exposed to any users of that new server.
The second use case is more traditional. Hard drives commonly fail, and drive manufacturers and resellers will often swap failing warrantied drives for new ones. This process is normally called “return merchandise authorization” or RMA. If a storage cluster has a large batch of failing drives, the RMA process can provide major cost savings over repurchasing new drives. However, if there is user data on the drives, the RMA process would result in that user data being exposed to unauthorized parties. The at-rest encryption feature in Swift is designed to protect against this sort of data disclosure. When using encryption in Swift, the cluster operator can confidently RMA drives.
How it works
Remember that Swift has a two-tier architecture: there’s a proxy server that handles most of the API and coordinates requests to the storage nodes, and there’s the storage nodes which actually persist the data. Swift’s encryption is implemented completely in the proxy server. If you’re deploying separate physical proxy and storage servers, this design allows for all of the crypto knowledge to stay on the proxy server and not even be on the servers where the data is persisted at all.
When the proxy server receives a write request, encrypts the data using AES-256 in counter mode. This raises some questions: where does the key come from and why counter mode?
Each object stored in Swift is encrypted with it’s own unique, randomly-chosen key. Internally, we call this the “body key”. This randomly chosen body key is itself encrypted with the object’s derived key. This is a technique called “key wrapping” where one key is encrypted with another key. The derived key is the HMAC (keyed-hash message authentication code) of the cluster’s master key and the full path to the object. The cluster’s master key is available to the proxy server. It’s very important that this master key is protected from untrusted parties.
Since we’re using key wrapping, it means that any eventual support for rotating encryption keys will not require re-encrypting the entire object contents. We’ll only have to re-encrypt the body key. Although we do not support key rotation yet, we wanted to build the foundation for it as part of our initial work on the feature.
We use AES in counter mode because it has the property that byte offsets in the plain text and the cipher text are the same. This allows us to easily support range requests to encrypted data. One of the goals of at-rest encryption in Swift is that it be completely transparent to the end user. All existing API functions should work in exactly the same way for both encrypted data and non-encrypted data.
Encryption is available in SwiftStack
Encryption is available for all SwiftStack customers and is easily enabled for all new clusters. As always, it’s free to try in your own environment at https://www.swiftstack.com/try-it-now. Also, if you’d like help adding at-rest data encryption capabilities, please feel free to reach out to us.