Latest in AI
Baseten Autoscaling Schedules: How to Prepare AI Models for Traffic Spikes
Scheduled replica settings move preparation ahead of predictable traffic, but the start time is not a readiness guarantee.
Baseten announced autoscaling schedules on October 30, letting teams change model-serving capacity before predictable traffic arrives. The feature supports recurring and one-time windows through its dashboard, CLI and Management API, reducing the need for an external timer to prepare replicas ahead of a busy period.
How Baseten autoscaling schedules work
The announcement describes time windows with minimum and maximum replica counts. A replica is a running copy of the model server. Raising the minimum ahead of demand gives additional copies time to start before requests need them.
This changes when capacity preparation begins. Reactive autoscaling responds to traffic that has arrived; a schedule uses something the operator already knows about the next peak.
The configuration guide specifies daily, hourly and one-time schedules, with defaults taking over outside active windows.
| Schedule rule | Current documented limit |
|---|---|
| Schedules per environment | Up to 10 |
| Minimum window | 5 minutes |
| Overlapping windows | Not allowed |
| Timezone | One shared timezone per environment |
| One-time duration | At most 48 hours, including consecutive chains |
Setup lives under Dedicated Inference: select the model and environment's deployment, open Configure autoscaling, then choose Schedules. Editing a currently active schedule applies immediately, not at its next recurrence.
There is an important qualification: a scheduled start changes settings; it does not promise ready replicas. A deployment already loading, migrating or undergoing a rolling promotion can delay application of the change. Recurring windows also cannot occupy the entire week, leaving time for the default configuration.
How early should a schedule start?
Baseten's startup-time documentation recommends using the p99 startup duration as the lead time before known spikes. That is more useful than choosing an arbitrary five-minute buffer.
Startup includes acquiring resources, obtaining the container and weights, and loading the model until it passes readiness checks. These phases can behave differently: a cached image can arrive quickly while model loading still takes time on every replica.
The platform's Startup time card separates these phases and shows percentile measurements. A wide gap between typical and slow startups is a reason to inspect the slow cases, rather than schedule around the average and assume the remaining requests will be fine.
The cost and reliability tradeoff
Consider a product that expects a surge immediately after a scheduled customer demonstration. Starting capacity at the demonstration's opening time may already be too late. Starting earlier creates a preparation window, but it also means keeping capacity available before useful work arrives.
That tradeoff should be measured against the cost of delayed requests, not treated as an automatic saving. A schedule based on an optimistic traffic forecast can leave more replicas running than needed; an underestimated maximum can still leave the service short during the peak.
Predictable demand is the best fit. Unexpected bursts still require sensible reactive settings, and schedules do not replace work on slow model initialization. The useful operational change is making the traffic forecast explicit in deployment configuration, where a team can review it alongside the capacity it will actually request.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help technical founders explain infrastructure improvements and product launches in content developers and founders can use and share.