Latest in AI
Modal Spindle: A Tinker-Compatible Training API With GPU-Time Pricing
Modal's open-source training server keeps the familiar API while moving performance tuning and compute economics into your workspace.
Modal announced Spindle on October 22, an open-source, Tinker-compatible training server that runs in your Modal workspace. It lets teams keep a familiar training interface while paying for GPU time rather than tokens. The opportunity is more control over costs and implementation, with more responsibility for using that compute well.
The launch announcement positions Spindle between a high-level training library and raw infrastructure. Developers write their own training loop, losses and rewards. Modal's documentation labels the service beta, so this should be evaluated as a new training option rather than treated as a settled production default.
What Tinker compatibility actually means
The Spindle repository distinguishes two paths. LoRA training uses the ordinary Tinker client API: multiple adapters can share a base model while keeping separate gradients and optimizer state. Full-parameter training gives each run dedicated training containers and uses additional Spindle helpers.
That distinction qualifies the headline promise of unchanged scripts. The familiar LoRA workflow can point at a different base URL and API key. Full-parameter training is an extension, rather than evidence that every script and feature is interchangeable with hosted Tinker.
The repository requires Python 3.12 and provides a version-pinned installation path. Its first versioned release, 0.2.0, is dated October 20. The October 22 announcement is the public launch being covered here, not a claim that no code existed earlier.
Why GPU-time pricing changes the calculation
Modal reports 20% to 80% lower effective cost per output token in selected comparisons with Tinker. Those are Modal's workload-specific measurements, not a new fixed per-token price or an independently verified saving for every team.
The mechanism is shared utilization. While one experiment waits for generated responses, another can use the trainer. This can make several experiments cheaper collectively, but it also makes scheduling and throughput part of the cost equation.
For a team running a steady stream of experiments, unused capacity is something it may be able to reduce. For a team running one short experiment occasionally, the same freedom might not justify the setup and tuning work. Compare the whole run, including startup, inference, checkpointing and retries, rather than converting a peak throughput number into an assumed bill.
Deploying your own server still involves setup
The setup guide describes deploying a Modal app, configuring authentication and pointing the client at the resulting server URL. GPU allocation starts on demand, and the first request can incur model loading and compilation time.
The server's compute allocation, backend and inference settings are configurable. That is the practical meaning of owning this implementation: teams can adjust the system beneath the API instead of waiting for a hosted provider to expose another setting.
It does not mean the infrastructure disappears. An experiment's desired concurrency and an account's available capacity still need to fit together.
Full fine-tuning has important operational limits
The full fine-tuning guide explicitly warns about GPU preemption and a 24-hour container lifetime. It calls for regular checkpoints and manual recovery from the latest completed checkpoint if in-memory training state is lost.
It also distinguishes checkpointing from publishing weights to a sampler. The former preserves training and optimizer state; the latter makes updated weights available for generating responses. Confusing those operations can leave a run without the recovery point its owner expected.
API compatibility reduces migration work, not operational accountability. A useful evaluation should show that the same experiment remains reproducible, survives the failures the documentation describes and reaches the desired result at a lower total cost. That is a stronger reason to adopt Spindle than the largest percentage in its launch benchmarks.
I'm Paras. I work at the intersection of AI, engineering and distribution.
If you're building an AI product, I help turn technical launches and product thinking into content developers and founders can understand and share.