Latest in AI
Baseten Bison and Groundhog: Test Long-Context Decisions Before Switching
Use the new System One models for long inputs, compare full and excerpted evidence, and recalibrate routing thresholds before shipping.
Baseten released bison-1 and groundhog-1 on November 20. Both are Gemma 4-based decision models that take text or images and return structured answers. The practical change for an existing Baseten integration is an eightfold increase in the documented context allowance, without switching away from the System One endpoint.
That is useful if a routing mistake comes from evidence your old input limit forced you to cut. It is not a reason to send every document on every request.
Keep the question and change the model
The decision-model guide lists 131,072 tokens per question for both new models, versus 16,384 for Mammoth, Elephant and Joey. The per-question limit includes shared input, so it is not a promise that you can submit a 131,072-token document plus more instructions and media.
Use POST https://inference.baseten.co/v1/systemone and the Bearer authentication shown in the API reference. The direct model slugs are:
decision-models/baseten-bison-latestdecision-models/baseten-groundhog-latest
Send the shared evidence in state and name each question in questions. Keep your existing schema for the first comparison. If you change the criteria and model at the same time, a better result will not tell you which change helped.
Check that extra evidence changes the right answer
Take an internal software support case where a new ticket calls an issue a bug, but the attached conversation shows a scheduled maintenance window. Try the same route question with the latest message, your current retrieval excerpt, and the fuller history. Use approved or synthetic examples for this test.
A long-input model should recover the maintenance detail in the full version. Also include cases where more history contains stale or contradictory information. Grade whether the decision used the relevant evidence, not simply whether it changed.
If the narrow excerpt already gets the job done, keep it. A system that retrieves a few relevant paragraphs may still be cheaper, faster and easier to audit than one that resends a full history.
Test images as a separate input path
For a screenshot decision, the System One reference puts image parts in a user message within state.messages. Do not carry over a Chat Completions request body unchanged or put an image in an assistant message; the reference says the latter fails validation.
Test one screenshot that makes the route unambiguous and another that does not. An ambiguous capture should reach a review path, not become the model's invitation to guess. If missing attachments are common, test that case separately instead of combining it with low-quality images.
Recalibrate before an answer becomes an action
The API guide distinguishes a choice answer and its option probabilities, a noul yes/no probability, and an ordered score response. Your application still owns the action. For example, it can suggest a queue and flag uncertain cases for review without allowing the API response to change any ticket.
Compare accuracy and the fraction of cases routed automatically at the thresholds you care about. Do not reuse a cutoff just because the new response has the same fields. A threshold that kept errors low for short text might behave differently on long conversations or screenshots.
Compare the bill at the same quality bar
The launch post lists $0.10 and $0.035 per million tokens for bison-1 and groundhog-1, respectively. At the stated rates, ten million billable tokens would be $1.00 and $0.35 respectively, before any tier differences or other charges.
Log actual usage, latency and review rates on the same request set. Also compare several related questions in one request against separate calls; the API can reuse a shared input prefix. That does not make extra questions free or mean their answers jointly satisfy your workflow.
Start with the lower-cost route if it meets your quality target. Move to the other only if the improvement justifies it, and keep the review path available when either model is uncertain.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help AI teams turn model launches and API changes into technical writing developers can use.