We are extremely excited to announce two major multi-region storage capabilities in SwiftStack.
First, something we call deterministic placement. Now policies can be extremely granular about exactly how many replicas each region should contain. (Of course, SwiftStack will continue to support a ‘as unique as possible’ distribution strategy to leverage all available capacity.)
Next, we are introducing multi-region erasure codes. Erasure coding is a space-efficient method for protecting data against failures. The SwiftStack approach is unique, using multi-region erasure coding to: 1) deliver optimal performance, 2) place minimal demand on the replication networks and 3) leverage all regions to amplify data durability.
Building on a Foundation
One of the key features and a differentiator of SwiftStack is our multi-region capabilities. A SwiftStack multi-region cluster spans geographically dispersed data centers while maintaining a single namespace. Data ingested in one region is automatically replicated and available in the other regions. If one region becomes unavailable then data is automatically served to the clients/applications from the other region. Since this is a single namespace, clients/applications do not have to change the way they access their data.
Over half of our customers deploy multi-region clusters. Some use it to store an offsite copy of their backup. Some use it to get data over to a DR site. Others use it because they have workflows that span multiple regions and they want to make the data available in the region that it is needed in. The point is that multi-region clusters are powerful and enable a whole slew of use cases.
Deterministic Placement of Data
By default, data in a SwiftStack cluster will be placed evenly across all the available capacity. This is best illustrated with an example. Let’s say you have two data centers – a primary data center for your active data, and a secondary data for a backup copy of your data. In addition, you want two copies of your data in your primary data center, and one copy in your secondary data center.
In order to support such use cases, we have introduced the ability to specify how many replicas of the data you want in each region. For the example above, you would define a storage policy that would specify that two replicas should be placed in the primary data center and one replica in the secondary data center. Once you apply this policy, SwiftStack will execute as commanded. It is as simple as that.
We also support more than one policy per cluster. So you could have a three-replica policy across two regions (such as the example above) and another three-replica policy across three regions. This gives you the flexibility to address different use cases/applications in the same cluster.
Multi-Region Erasure Coding
Erasure coding is a method for protection of data in which the data is broken up into fragments, and then encoded with redundant (parity) fragments. Erasure coding is generally a more space-efficient method (vs replication) for protecting data in storage systems.
SwiftStack has supported erasure coding as an option since SwiftStack 3. In fact, you could have both replica storage policies and erasure coding storage policies in the same cluster and apply them to different containers/buckets depending on the requirements of the client/application.
Now, erasure coding policies can be applied to multi-region clusters. One of our guiding principles for implementation of erasure coding in multi-region clusters was to provide optimal performance while placing minimal demands on the replication network. We designed the implementation so that you can sustain the loss of a node (at minimum) or multiple drives and still be able to serve the object without having to retrieve fragments from other regions.
So instead of erasure coding a single copy of the data and spreading the resulting data and parity bits across all available regions, SwiftStack will erasure code entire copies of the data in each region. This minimizes the occasions that fragments need to be retrieved from another region and traverse the replication network in order to serve up a request for the data. This allows for multi-region erasure coding to be used in environments that require high levels of performance, but do not have high throughput and low latency connections between regions.
Finally, because the storage policy is part of the same cluster, all the data protection bits are used to amplify its durability. This strategy provides extremely high levels of durability for data.
Multi-Region Erasure Coding
Multi-region clusters are a great way to protect your data, enable multi-region workflows and collaborate across regions. SwiftStack’s implementation is flexible and extremely easy to implement using our Storage Policies. Please reach out to learn more and we can help plan your deployment.