# Together Storage Accelerator: Test Cold Reads, Warm Reads and Checkpoint Correctness

Set up an isolated cache trial and measure whether it cuts GPU startup time without serving outdated checkpoint files.

By Paras Madan | 2026-11-21


[Together AI added Storage Accelerator](https://www.together.ai/blog/introducing-together-storage-accelerator) to its GPU clusters on November 19. It caches reads from S3-compatible storage or Hugging Face Hub on cluster NVMe. If a dozen workers pull the same checkpoint, the cache can save them from downloading it from the origin a dozen times. But the warm-read headline won't tell you how long the first job waits, or whether the bytes your next job loads are still current.

## Choose one repeated read to test

Pick one model checkpoint or dataset shard that several workers already use. Keep your existing origin as the source of truth, and start with a versioned test object that has the same shape and size as the real workload. An object that each worker downloads once and never sees again isn't a useful test of the cache.

The add-on has no separate charge on Together GPU Clusters. That doesn't make GPU time, origin storage or egress free. The [setup guide](https://docs.together.ai/docs/storage-accelerator#enable-the-add-on) notes that the TSA control may not be visible until your organization has access. Check the cluster settings before changing your workload.

## Change one reader, not the whole application

Enable TSA and add a backend with read-only access to the test data. For S3, the client uses path-style addressing and includes the backend name before the bucket and object key. Setting `AWS_ENDPOINT_URL` alone is not a complete migration if your existing bucket and key no longer map to that path. Test with one loader before changing the environment for every job.

Keep the existing upload path. The cache is intended for reads; listings still go to the origin and a plain write request can reach the origin if the backend credentials allow it. The [TSA security notes](https://docs.together.ai/docs/storage-accelerator#security) also explain that cluster workloads can reach configured backends. Don't rely on independent application credentials to do the isolation for you. Use a dedicated test scope and verify access restrictions before using a shared cluster.

## Measure cold and warm startup separately

Measure a direct read first. Then use an uncached object through TSA for a cold measurement and repeat the same read for a warm one. Run from a second worker as well. Track wall time until the model actually loads, bytes from the origin and whether the data matched your reference.

On one H100 node in us-central, Together reported 12.7 to 15.0 seconds for a direct 2 GiB R2 read and 0.64 to 0.72 seconds from warm cache. The underlying [performance table](https://docs.together.ai/docs/storage-accelerator#performance) puts the cold cached-path read at 16 seconds and says the benchmark did not measure peer-node reads. Don't turn the warm single-node result into a prediction for every node or a promise about your whole job.

## Give checkpoints immutable names

Avoid overwriting a path such as `latest/model.safetensors`. An S3 object changed at the same key can still be served from cache, and a read that spans an overwrite can mix old and new bytes. Use a unique key for each checkpoint and have the job select that exact key. Keep a manifest with the expected digest outside the cache trial and compare the downloaded bytes before loading them.

For Hugging Face, the add-on's mirror requires a configured token, and resolves branches and tags to commits before caching. It is still worth pinning a revision in tests so all workers are comparing the same inputs. A successful model startup alone doesn't prove they loaded the intended checkpoint.

## Decide whether the cache earns its place

Run a normal restart and a many-worker restart. Track the slowest worker's time to readiness, not just the fastest download. If origin requests drop but every worker still takes just as long to load the model, you may save egress without saving usable GPU time.

I'd roll this out where restarts are frequent and the same immutable data is shared. If the job reads mostly new objects or rewrites the same keys, fix the data layout first. A faster read of the wrong checkpoint is worse than waiting for a fresh one.

I'm Paras Madan. I write about AI products and how engineering teams put them to work.

If you're launching infrastructure and need a guide that explains what to try and what to measure, I can help with the writing.

[Book a 30-minute call](https://cal.com/paras-madan/30min).
