This week about twenty of the top contributors to Swift gathered in Longmont, Colorado to spend a few days hacking on Swift, discussing new features, and talking in person. It was a very productive week, and we are all grateful to Paul Luse at Intel for sponsoring it. We had attendees from SwiftStack, Rackspace, HP, Seagate, eVault, NTT, Red Hat, eNovance, Box, and Intel (and flying in from Japan, Germany, Ireland, UK, USA, and China).

As you may know, we have some major features in progress in Swift right now. While our time together certainly involved more than just work on storage policies, we did focus on the last bits necessary to get the feature merged into Swift.

june_denver_hackathonIn addition to going over the proposed patches, as a group we were able to improve the storage policy implementation in a few significant ways. My favorite example is how the upgrade process was made better to ensure Swift continues to play well from a operator perspective. The storage policies changes do involve some database schema changes. As we’ve always done in Swift, we have a seamless upgrade path for this. However, the implementation prevented any downgrades, and this line was crossed as soon as a deployer installed the new code. Through the discussion about this, Chuck Thier proposed the idea of using SQLite’s updateable views to allow seamless downgrades from storage policy code. This change makes rolling upgrades easier and gives deployers the ability to upgrade the code and, if necessary, roll it back without customer impact as long as additional storage policies have not been defined.

We’re nearly done with storage policies, and with the last few changes from this week and the last few reviews needed for the code, I expect that we’ll be able to merge the feature into master next week. I’ll be sharing more info about that as we get closer to the release.

Our initial reason for building the storage policies feature into Swift was so that we can support erasure coded data in Swift. At the hackathon this week, we also spent quite a bit of time reviewing the design for erasure coded (EC) data in Swift. These conversations helped share understanding of how the feature will work and raised many questions that have yet to be answered. I’m excited about the progress we’ve made, and I’m looking forward to having an EC storage policy in Swift.

I’ve already seen a lot of interest in the community on how to use storage policies to integrate with Swift. Red Hat has created a project called SwiftOnFile that allows deployers to run Swift on top of existing file systems. The SwiftOnFile project currently is currently used to enable GlusterFS as a storage policy in Swift.

Other topics discussed during the week include integration with Keystone v3, review of a few outstanding patches, and a multi-hour retrospective on using long-running feature branches in Swift. We finished up the week with a discussion of performance in Swift and ways to improve it. I expect that each of these topics will lead to improvements in Swift that will be seen in the next several months.

Overall it was a great week for the Swift community and a great week for progress on OpenStack Swift itself. I’m already looking forward to the next time we can all get together in a room again.

About Author

John Dickinson

John Dickinson

John joined SwiftStack as Director of Technology coming from Rackspace where he worked on Rackspace Cloud Files and OpenStack Swift, the large-scale distributed object storage system which SwiftStack is built on. As the Project Technical Lead (PTL) for OpenStack Swift, John is also leading much of its community development efforts.