SHARE   

It has not always been easy to test for compatibility with object storage systems during application development or workflow design. From an accessibility standpoint, one of object storage’s big advantages is that it is API-driven over HTTPS, making it possible to be accessed  from anywhere, but this is sometimes not an option if the storage functionality needs to be behind the firewall.

In this article, we’ll walk through some of the available options for testing object storage and what we’re doing at SwiftStack to help.

#1: Use a Public Object Storage Service

The quickest and easiest way to test object storage is to use a public service. All that needs to be done is to create an account on that service, create a bucket to hold data, and feel the credentials and path into the application.

Advantages

Easy to configure, as the infrastructure is already deployed and running. You also never need to worry about maintaining this infrastructure, so it will be there (most of the time) without you having to invest any energy into it. It’s also free to use for a period of time if you keep usage levels low.

Disadvantages

To use a public service, applications need to be able to communicate with it. If the apps are running on-premises, and this is not possible, using a public service is not an option. Even if you can get access, if you app is far away from the service, performance issues could make it prohibitive. Also, after a certain period of time or if usage crosses a defined threshold, costs can get high due to storage and egress charges.

To use a public service, applications need to be able to communicate with it. If the apps are running on-premises, and this is not possible, using a public service is not an option. Even if you can get access, if you app is far away from the service, performance issues could make it prohibitive. Also, after a certain period of time or if usage crosses a defined threshold, costs can get high due to storage and egress charges.

Options

  • SwiftStack option: The SwiftStack API Endpoint gives you the ability to test your application with a complete SwiftStack cluster running in our data center. We will provide you credentials to your own bucket that is accessible over the internet. This option is excellent if you want to quickly test compatibility with the S3 or Swift APIs and want to ensure your app works with SwiftStack. Using a SwiftStack-provided API endpoint is always free for testing purposes.
  • Other popular options: Amazon S3, Google Cloud Storage.

#2: Run a Lightweight Tool to Test the S3 API

If your application or workflow is on-premises, it may likely be optimal to also have the storage service local. To accomplish this in the past, you used to have to buy and deploy an expensive, production-level storage system, even if you are only in the development or test phases. Now, there are lightweight software tools that help you test object storage APIs without having to deploy a full storage stack.

Advantages

This option only requires a minimal footprint, as it can be deployed on your laptop or existing Docker cluster, and it can be up-and-running in minutes. Since it does not require a connection to outside services, your application will experience local performance characteristics and it eliminates all of the disadvantages to using a public service during the development and test phases.

Disadvantages

Is not a complete storage stack or may not be built using the same code as the storage solution your app will eventually be deployed with. It is limited by the storage resources underneath it, so it may not be ideal for high capacity or extensive infrastructure performance or stress testing.

Options

  • SwiftStack option: pico swiftstack is a Docker image allowing you to quickly run the access layer of SwiftStack so you can test for S3 or Swift API compatibility. pico swiftstack may also be used when integrating with a CI/CD system. If your application works with pico swiftstack, it will be compatible with the standard APIs and the complete SwiftStack platform.
  • Other popular options: s3proxy, Adobe S3mock, MinIO.

#3: Install a Complete On-Premises Storage Stack

Most storage vendors now claim their product can be downloaded and freely tried on your own hardware. While this sounds great, it is not quite a reality across the industry yet and is presented this way mostly as a sales tool (blah).

Advantages

Deploying the complete storage stack in your environment will not only ensure you applications are compatible, but it will allow you to feel how it is deployed, managed, scaled, and how it performs. Then, when you go from test to production, the number of potential surprises will be minimized.

Disadvantages

To deploy a software-defined storage system on your own infrastructure, there are a few requirements that the prior two options do not have. First, you must have available servers to run the software on. Since this is storage, deploying in a VM environment is not optimal. To be able to run on any hardware, the software runs on top of the operating system (most often Linux), so you will need basic Linux stills like installation and networking. The steps to get the system up-and-running are easy, you just need the resources.

Options

  • SwiftStack option: The complete SwiftStack platform is free to download and install and all of our documentation is open to read. It starts by creating an account in the SwiftStack Portal, which will give you access to the software. Administration of this environment is performed using the hosted as-a-service SwiftStack Controller, which simplifies the deployment steps. Not only does this allow you to test object storage APIs, but also SwiftStack’s file access interfaces and SwiftStack 1space multi-cloud data management.
  • Other popular options: RedHat Ceph, MinIO.

Unlike in the past, to test object storage you do not need to interact with an enterprise sales team, procure costly infrastructure, or even interact with central IT. This will help with the more rapid development of data-driven applications and workflow.

Not only is SwiftStack freely available to try, with many options, but is backed by a solid technical staff and subject matter experts. If you would like assistance with trying out object storage or to have a conversation about best practices, please feel free to reach out.

About Author

Erik Pounds

Erik Pounds

Erik is an avid technology geek, attacks opportunities by building things, and currently leads the marketing function at SwiftStack. Prior to SwiftStack, he led the Sync team at BitTorrent, ran product management at Drobo, and held various product and marketing roles at Brocade and EMC. He proudly graduated from the University of San Francisco, where he captained their Division 1 Golf Team.