1space Cloud Connector

1space Cloud Connector lets public cloud compute infrastructure access the data in a 1space mapping. Applications authenticate their S3-API requests against 1space Cloud Connector just as they would against the S3 API of the SwiftStack cluster.

The 1space Cloud Connector is distributed as a container image and requires the ability to run containers.

Features

If you configure a 1space Cloud Connector configuration and push a config, then:

  1. The relevant config files will be created and PUT into the configuration's specified configuration bucket and prefix. This location is in the S3-API object store that is "local" to the public cloud compute infrastructure.
  2. Every 1space Cloud Connector container will automatically reload its 1space mapping and S3 user database after a SwiftStack cluster config push. Other configuration changes require you to restart the containers.
  3. Client S3 requests use the same signing credentials they would if the client was talking directly to the SwiftStack cluster.
  4. Client requests are in terms of the 1space mapping's "Local Bucket/Container". So the bucket in the 1space Cloud Connector S3-API requests will be a container in the SwiftStack cluster. Again, this is just as if the client request were being made to the SwiftStack cluster using the S3 API.
  5. GET and HEAD object requests read from the "local" S3-API object store if possible, but fall back to read from the "remote" SwiftStack cluster.
  6. PUT object requests write to the "local" S3-API object store.
  7. Object metadata updates are supported.
  8. S3 Multipart Uploads are supported.
  9. Bucket listings are merged--they show objects in the "local" S3-API object store as well as the SwiftStack cluster.
  10. You can run as many 1space Cloud Connector container instances as you want--they share nothing, and scale horizontally.
  11. The Docker container image for the 1space Cloud Connector is publically available from Docker Hub as swiftstack/cloud-connector:TAG, where TAG is a release tag. The SwiftStack Controller will display its current cloud-connector release tag on the cloud-connector configuration list page.

Limitations

  1. Only the S3 API is supported at this time.
  2. There is no fine-grained access control; the S3-API credential used to access cloud-connector has access to any 1space mapping involving any container within the one default Swift account associated with the S3 credential. Likewise, a mapping involving containers in one account cannot be accessed using S3-API credentials of a user whose one default Swift account is a different account.
  3. Account-level requests are not supported.
  4. Only GET and HEAD requests are supported for buckets (containers).
  5. 1space Cloud Connector only speaks HTTP; if TLS (HTTPS) is desired, a TLS-terminator must be deployed in front of cloud-connector.
  6. Any desired load-balancing must be deployed in front of 1space Cloud Connector.
  7. When 1space Cloud Connector configurations are deleted, the config files are not deleted from the S3-API object store.

Configuration Example

The following example shows how to configure and run 1space Cloud Connector. This example uses Amazon services, but you can use any docker runtime as long as you configure it similarly with respect to networking and permissions.

Bucket and Permissions For 1space Cloud Connector

Create or choose an existing bucket to hold cloud-connector configuration. This example will use a bucket called dockertest2.

../_images/cloud_sync_cloud_connector_aws_bucket_list.png

Choose a prefix within the bucket under which the controller will store 1space Cloud Connector config files. We recommend you end this prefix with a forward slash (/).'

Create or select an IAM Policy that only has permissions into the configured prefix in the configuration bucket. Here is an example JSON IAM Policy for a bucket dockertest2 and a prefix of a_config_prefix:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::dockertest2/a_config_prefix/*",
                "arn:aws:s3:::dockertest2"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::dockertest2",
            "Condition": {
                "ForAllValues:StringLike": {
                    "s3:prefix": "a_config_prefix/*"
                }
            }
        }
    ]
}

Associate that IAM Policy with a Group and a User, then save that user's credentials in a secure location. These are the credentials you will enter into the controller and associate with the 1space Cloud Connector configuration. These should be different credentials than you use for any 1space mappings.

Create a 1space Cloud Connector Configuration

Create a 1space Cloud Connector configuration in the SwiftStack Controller. Each configuration specifies where to store the 1space Cloud Connector configuration files ("Configuration Bucket Name" and "Configuration Prefix"), what S3 credential set to use to store the configuration files, and what base URL should be used by cloud-connector to connect to the SwiftStack cluster.

You should click "Verify" to ensure the chosen credentials have access to the specified bucket and prefix combination.

../_images/cloud_sync_cloud_connector_add.png

Note Docker Env Vars In List

The list of 1space Cloud Connector configurations includes the list of Docker environment variables you will need to include when you run a 1space Cloud Connector container for that configuration. You can also edit and delete 1space Cloud Connector configurations from this list.

../_images/cloud_sync_cloud_connector_list.png

If you use Amazon ECS to run 1space Cloud Connector containers, you can associate the 1space Cloud Connector configuration IAM Policy with an IAM Role and associate that Role with the container. Once you do that, you do not need to provide the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. This is the Task Role in the Amazon ECS Task Definition edit page:

../_images/cloud_sync_cloud_connector_aws_task.png

Push Cluster Config

../_images/cloud_sync_cloud_connector_push_config.png