SwiftStack Blog

TensorFlow 2.0 and Object Storage: Train Neural Networks with Large Datasets

This past summer, I shared a tutorial for how to load large datasets into TensorFlow 1.x from object storage using the S3 API. This new tutorial will show how TensorFlow 2.0 can leverage data from an object store. For those unfamiliar, object storage is the de-facto standard for storing unstructured data in environments requiring massive […]

How I Used pico swiftstack to Test TensorFlow with S3 Storage

A few weeks ago, I put together a tutorial on how to get S3 data into TensorFlow and I used pico swiftstack as an example data source. This article goes into a little more detail about what pico swiftstack is and how I used it to quickly test the S3 API. First off, pico swiftstack […]

Tutorial: Loading Data into TensorFlow via S3 API

Github Gist/Google Colab Link When I was looking for code examples of loading data into TensorFlow via the S3 API, the results were pretty sparse (with lots of questions about various error messages).  I thought to myself, “someone should really write a tutorial on using S3 data in TensorFlow!” Then, I realized that someone was […]