SHARE   

Using Aspera with SwiftStack private cloud object storage unlocks numerous possibilities, including architectures that are not possible with traditional NAS/SAN storage.  These advantages include:

  • Integrated disaster-recovery: multi-region functionality can write to 2+ physical locations simultaneously and transparently fail-over

  • Resilient: read from and write to storage despite poor network conditions

  • Ultimate DMZ: immune to ransomware; fully encrypted in transit and at rest

  • Native format: data written by Aspera can be immediately read by other applications

  • Interoperable: Aspera can read and accelerate pre-existing data written by other apps

  • Trusted: use the M&E industry-standard tool to get data in and out of SwiftStack

  • Ingest/delivery buffer: relieve stress on traditional NAS/SAN by staging in SwiftStack

  • Empower your active archive: turn it into your CDN Origin!

A note about the author: I architected, administered, and operated Aspera as a customer for 6+ years prior to joining the SwiftStack team.  As one of the first joint SwiftStack and Aspera customers, the instructions in this blog (and detailed integration guide) come from 2+ years of experience running Aspera and SwiftStack together in production at many-petabyte scale.

 

Aspera + SwiftStack Use-Cases

Traditional transfer acceleration: Aspera can be used to allow external parties to transfer directly to/from SwiftStack, usually via the public internet.Aspera accelerates data transfer in and out

Internal data-mover: Aspera can be used to move data between your existing NAS/SAN and SwiftStack.Aspera moves data between storage tiers

Integration Steps

Prerequisites for this solution are:

  • Aspera High-Speed Transfer Server (formerly Enterprise Server) running on Linux
  • A valid SSL certificate on your SwiftStack cluster
  • A new or existing account and bucket on your SwiftStack cluster that you’d like to use with Aspera


For help setting up a new Aspera server on Linux, refer to Aspera’s documentation on the following page for your preferred Linux distribution:
https://downloads.asperasoft.com/en/documentation/1

For help setting up a new SwiftStack cluster, please refer to the SwiftStack QuickStart Guide or reach out to us for assistance: https://www.swiftstack.com/docs/install/index.html or info@swiftstack.com

The steps below includes instructions for both the S3 API and OpenStack Swift API.  While Aspera supports both APIs, there is little reason to use both simultaneously other than for testing purposes.  For help picking an API and a detailed list of pros and cons for both APIs, see our detailed integration guide—otherwise, we generally recommend using the S3 API.

Step 1: Enable Aspera TrapD service

Run $ /opt/aspera/bin/astrap-config.sh enable

(https://download.asperasoft.com/download/docs/entsrv/3.9.1/es_admin_linux/webhelp/index.html#dita-trap/trap_intro.html)

Step 2: Modify s3.properties or swift.properties file and restart services

Open `/opt/aspera/etc/trapd/s3.properties` in your favorite text editor and uncomment and modify the following lines to match:

s3service.https-only=true

s3service.s3-endpoint=<your SwiftStack cluster’s S3 API URL>

s3service.disable-dns-buckets=true

s3service.default-bucket-location=<your SwiftStack cluster’s S3 API region>

OR

Open `/opt/aspera/etc/trapd/swift.properties` in your favorite text editor and uncomment and modify the following lines to match:

aspera.swift.endpoint.auth-path=/auth/v1.0

aspera.swift.https-only=true

Restart Aspera services (this will interrupt in-progress transfers)

$ service asperanoded restart

$ service asperacentral restart

$ service asperanoded restart

Step 3: Edit docroot (and transfer permissions if necessary) for new or existing Aspera user using asconfigurator

S3 API
$ asconfigurator -x "set_user_data;user_name,<Aspera-username>;absolute,s3://<SwiftStack-user>:<S3-API-Key>@<SwiftStack-S3API-URL>/<bucket>

    1. Example asconfigurator command that uses the following information for the S3 API:
      • Aspera user: “swiftstack-asp-s3api
      • SwiftStack account: “aspera”
      • S3 API Key for account: dc64d1222079e556xd3b36b6bae370a
      • Bucket in account: “asperabucket”
      • SwiftStack S3 API URL: “sestorage.swiftstack.tech”

$ asconfigurator -x "set_user_data;user_name,swiftstack-asp-s3api;absolute,s3://aspera:dc64d1222079e556xd3b36b6bae370a@sestorage.swiftstack.tech/asperabucket"

Swift API (skip this if solely using S3 API)
$ asconfigurator -x "set_user_data;user_name,<Aspera-username>;absolute,swift://<SwiftStack-user>:<password>@<SwiftStack-URL>/<bucket>"

Example asconfigurator command that uses the following base information:

      • Aspera user: “swiftstack-asp-swift”
      • SwiftStack account: “aspera”
      • Password: aspera12345aspera
      • Bucket in account: “asperabucket”
      • SwiftStack URL: “sestorage.swiftstack.tech”

$ asconfigurator -x "set_user_data;user_name,swiftstack-asp-swift;absolute,swift://aspera:aspera12345aspera@sestorage.swiftstack.tech/asperabucket"

For more information, see: https://download.asperasoft.com/download/docs/entsrv/3.9.1/es_admin_linux/webhelp/index.html#dita-trap/object_storage_docroots.html

 

You’re now set up to use Aspera to transfer data in and out of SwiftStack!


Whether to use the S3 API or OpenStack Swift API when integrating Aspera with SwiftStack is up to you.  For a detailed explanation of the differences between the two APIs when using Aspera, and for more information, please see our detailed Integration Guide.

Download Now

About Author

Vince Auletta

Vince Auletta

Vince Auletta is the Director of Media Solutions at SwiftStack and a lifelong fan of visual storytelling, technology, and the many ways they intersect. Based in Los Angeles, Vince focuses on helping SwiftStack's media clients get the most out of their solutions through a combination of on-site consultations and meticulous testing in SwiftStack's high performance Media Solutions Lab. Prior to joining SwiftStack, he was the Director of Technology & Security at Premiere Digital, where he helped pioneer the use of scale-out 'cloud' techniques and technologies on-premises for extremely high volume workflows. Vince holds a B.A. in Film and Media Studies from the University of California at Santa Barbara, where he honed his skills as a cinematographer and colorist during the transition from celluloid film to digital cinema cameras.