SHARE   

A few weeks ago, my colleague Jon Kelly posted a blog article titled “How I Used pico swiftstack to Test TensorFlow with S3 Storage”. Since pico swiftstack is a relatively new product, I would like to explain why having a containerized S3-compatible object storage option for testing and experimenting is helpful to anyone evaluating scale-out, software-driven storage.

SwiftStack is a highly-scalable distributed data storage and management platform, but before you use deploy it into production, you probably want to run it through some motions. Will it support all the functionality your application needs? How are the API responses structured, for you to parse? Can it spin in mid air with a torch between its teeth? Well, you must test that. But setting up the full platform to check application or workflow compatibility, even based on virtual machines, can add additional time and consume more physical resources than necessary. So, how can we make it easier?

Simple Option for Locally Testing S3 API Compatibility

One option we have had for a long time was for you to test against an instance of SwiftStack that we host. When requested, we can provision API endpoints for you or your developers to test against without having to set up any infrastructure. However, different developers have different needs, and having a local containerized instance, with no need to communicate over the Internet, is another option to test for object storage API (S3 and Swift) compatibility.

pico swiftstack

Containers have been around for a while, long before Docker came around and made them usable by a broad audience. Containers isolate applications and resources. They create a native confined space on a host, which looks and feels like a real server, and allow you to control their personality and the level of interaction they have with the real world. For example, you can create an Ubuntu container on a Red Hat server, choose if they get their own network stack or use the host’s, and much more.

So, we created a container-based SwiftStack and call it “pico swiftstack”. It is not meant to be run as a full storage system, just as an endpoint for developers to quickly instantiate the access layer of the SwiftStack platform locally and run a test suite against it.

Architecturally, pico swiftstack is running on Alpine Linux, which has been marked as the best OS to use for containers, and is a personal favorite of mine (just as a data point, moving pico swiftstack from an Ubuntu image to an Alpine image cut its size down by more than 50%). We use S6 process scheduler (via the excellent S6-overlay).

It runs all the SwiftStack object storage services along with SwiftStack Auth, and exposes both the S3 or Swift API for you to run any of the PUT, GET, POST, DELETE and other verbs. You can also connect to it with our SwiftStack Client, or any other S3 or Swift-compatible tool.

Our intention is to keep pico swiftstack on par with our SwiftStack platform release cycle, so that every release we make will have its own swiftstack/picoswiftstack:<release> for it. So, swiftstack/picoswiftstack:6.28.0.0 will use the same components as the full SwiftStack v6.28.0.0 product uses. swiftstack/picoswiftstack:latest will also point to our latest release. So, if you want to compare or troubleshoot something from the past, you can run specific SwiftStack version, and if not – just run latest.

You can find pico swiftstack, along with how-to instructions, in our Test Drive page. We are continuously adding functionality to pico swiftstack, and if you have any feedback on its capabilities, please let us know at pico@swiftstack.com.

Test Drive For Free

About Author

Ehud Kaldor

Ehud Kaldor

Ehud Kaldor started his path into storage in 2004, by joining a startup named Kashya, which was acquired by EMC in 2006 and became EMC RecoverPoint. At EMC, Ehud served as a CSE (corporate systems engineer) for RecoverPoint, and later led the CSE team for RecoverPoint. After 8 years at EMC, Ehud joined NetApp as an SE, and was part of some of NetApp's largest accounts' teams, working with cutting-edge customers. Ehud is a technology buff, and learns as much as possible on anything and everything. He dips his toes into container technology, programming, system administration, and likes to break stuff and put it back together. He is also an introverted extravert (or an extraverted introvert), which accounts for his ability to present to audiences along with his personal awkward social behavior. Ehud is currently an SE at SwiftStack.