Latest in AI
Baseten Mammoth-1 and Elephant-1: Decision Models, API and Pricing
Baseten's new models return probabilities instead of prose, giving developers another way to route requests and decide when to ask a human.
Baseten released Mammoth-1 and Elephant-1 on October 21, 2026. These decision models classify, route and score inputs without generating a written answer. Both have hosted APIs and downloadable weights, with a temporary free offer that makes them practical to evaluate for high-volume application decisions.
The useful question is whether a task needs a paragraph at all. A support workflow may only need to know which queue owns a ticket and whether someone should review it before routing.
What Mammoth-1 and Elephant-1 do
Baseten's release announcement describes two Qwen3.5-derived models trained for decision-making. Mammoth-1 has 27 billion parameters; Elephant-1 has 9 billion. Their outputs cover predefined choices, yes-or-no questions and ordered scores.
Instead of asking a chat model to write a label and then parsing its answer, an application receives probabilities for the permitted answers. Those probabilities can support a rule that routes clear cases automatically and sends ambiguous ones to a person.
A probability is useful only if it behaves reliably on your data. A model giving one option a high score does not, by itself, establish how often that decision will be correct in your application.
API access, context window and free pricing
Vercel's AI Gateway announcement confirms both models are available through its gateway, free through October 27. Both support a 16,000-token context window.
| Detail | Mammoth-1 | Elephant-1 |
|---|---|---|
| Parameters | 27B | 9B |
| Gateway model ID | baseten/mammoth-1 | baseten/elephant-1 |
| Context window | 16k tokens | 16k tokens |
| Gateway launch offer | Free through October 27 | Free through October 27 |
The gateway example uses the AI SDK's experimental decide function. A request supplies the input state and named questions, rather than a conventional chat conversation. Multiple questions can evaluate the same input in one call.
Baseten separately offers a free week through its Model APIs and states that zero data retention is enabled there by default. That statement applies to Baseten's service; it should not be assumed to describe every intermediary's data handling. The launch material does not establish a permanent post-promotion price.
How the models avoid repeated work
The Base Labs technical report explains that the models process shared context once and reuse its cache across questions. Each additional question then processes its own instructions and answer options.
The researchers also modify the full-attention layers so tokens can attend across their whole block. A question can see its answer options, while the shared prefix remains independent of later questions. The recurrent layers retain causal behavior.
This architecture fits workflows that ask several bounded questions about one document. For a support ticket, the same text could inform routing, urgency and whether a refund was requested. The application still owns the action that follows each answer.
What the benchmark claims do and do not show
Baseten reports that Mammoth-1 leads Jev Decision Index 0.4 with 71.24. Its technical report gives Artificial Analysis Decision Index scores of 65.9 for Mammoth-1 and 65.2 for Elephant-1, but labels those as its own replication of that evaluation. They should not be presented as independently verified hosted leaderboard results.
These are decision benchmarks, not evidence that either model replaces a general coding or conversational assistant. They also cannot tell you the error cost of sending a real customer's ticket to the wrong team.
The evaluation that matters is fairly concrete: compare accuracy, calibration, latency and cost per accepted decision on a held-out sample of your own work. Include the difficult cases and count human escalation, retries and fallback model calls.
A smaller decision model can make an application more efficient if it reliably handles the repeated choices around an expensive agent. The important boundary is the point where its answer becomes uncertain enough that the system stops acting automatically.
I’m Paras. I work at the intersection of AI, engineering and distribution.
I help technical founders explain model releases, API choices and engineering tradeoffs in content that developers can understand and share.