Baseten Serverless Loops: Training Costs, Checkpoints and Cleanup

Baseten made Serverless Loops generally available on November 6 for supervised fine-tuning and reinforcement learning. Training can now run without reserved GPUs, but deploying the result creates a separate billing decision. The useful question is how to experiment without accidentally leaving dedicated compute running.

The GA announcement confirms the release. This guide follows the documented workflow, rather than a first-hand training test, and focuses on choosing the billing mode, preserving a usable checkpoint and cleaning up the right resources.

Set the billing mode explicitly

Baseten's availability documentation distinguishes token-billed serverless runs from dedicated trainers and samplers billed for GPU time, including gaps between operations. Serverless therefore suits experiments where you stop to inspect results or rewrite the reward function. Dedicated capacity offers more control over replicas and configurations.

There is an important default: when serverless supports the selected model, LoRA rank and sequence length, Loops uses it. Otherwise, the default can select dedicated mode. Set scope="serverless" if token billing is a requirement. Unsupported configurations then fail instead of falling back to GPU billing.

For a first experiment, that failure is useful. It forces a deliberate choice between changing the configuration and accepting a different cost model. A script that starts successfully is not necessarily running under the budget assumptions that motivated it.

Check the model before copying a recipe

The current model catalog includes Qwen3.6-35B-A3B, GLM-5.3 and GLM-5.3-Flash, each listed at a maximum sequence length of 262,144 tokens. Qwen and GLM Flash support vision; GLM-5.3 is text-only, and its long-context configuration excludes cross_entropy and dpo losses. Workspace permissions can impose tighter limits than the catalog.

Loops trains LoRA adapters while leaving base weights frozen. If the plan requires full-parameter training or a different training framework, Baseten directs that work to Training Jobs. “Managed training” covers several different products here, so choose the training method before choosing the infrastructure.

Start with a small, measurable task. For example, a support-ticket classifier could reward the correct category while evaluating separately on tickets excluded from training. Keep the category definitions fixed during a comparison. Otherwise, a better-looking score may reflect an easier scoring rule rather than a more useful model.

Save two kinds of checkpoint

The official quickstart separates deployable sampler weights from resumable training state. save_weights_and_get_sampling_client() publishes a checkpoint for sampling and deployment, but omits optimizer state. Use save_state() when you need to continue training later.

Serverless sampling clients are pinned to the checkpoint used to create them. After another training update, create a new client for the newer weights rather than assuming an existing client follows the trainer automatically.

For the ticket classifier, record the base model, training run, checkpoint and held-out score together. Keep a baseline before training and compare it with the exact checkpoint being considered for deployment. A successful sample proves the endpoint works, not that training improved the task.

Treat deployment as a separate expense

The quickstart deploys a sampler checkpoint to dedicated inference, which bills for its GPU while live. Deactivating the Loops run does not delete that deployment. When finished, delete the deployment separately to stop its GPU billing. Closing local Python clients does not deactivate a training run either.

Keep three lines in the experiment budget: training and sampling usage, evaluation work, and the deployed endpoint. Per-token training rates alone cannot answer the total-cost question. This guide does not quote an unverified dollar rate or imply that serving the result uses the training tariff.

A sensible first milestone is a saved, evaluated checkpoint. Only provision the endpoint once there is something worth serving, and decide who owns shutting it down after the evaluation. Making deployment easy also makes it easy to create resources nobody remembers to remove.

Diagnose a refusal before retrying

Baseten's troubleshooting guide distinguishes a full shared pool from an organization limit. A capacity 429 may clear as capacity becomes available, but details.limit="org_cap" requires freeing a run or sampler, or obtaining a higher limit. Repeating the same request does not fix that limit.

An interrupted script also deserves inspection before a restart. An SDK timeout does not shut down the run. Retain the run and session IDs so you can find existing work rather than creating another experiment blindly.

Serverless training removes an infrastructure burden, but it does not remove experiment management. The strongest workflow makes billing mode, checkpoint identity and cleanup explicit enough that a failed experiment is cheap to abandon and a successful one is safe to reproduce.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams explain infrastructure, developer workflows and product launches in clear technical stories people can use.

Book a 30-minute call.