# Sarvam-105B-1711: Check the Serving Configuration Before Swapping Weights

Sarvam's updated checkpoint comes in NVFP4 and BF16. Plan the migration around parsers, thinking settings and measured capacity, not just benchmark scores.

By Paras Madan | 2026-11-18


Sarvam-105B-1711 is worth evaluating if you already serve Indian-language assistants or tool-using workflows. The useful question is not whether its benchmark table looks better. It is whether your application can move to the new checkpoint without silently changing how it interprets reasoning, calls tools or handles a busy queue.

The [official model card](https://huggingface.co/sarvamai/sarvam-105b-1711) describes further post-training of the same 105B base, targeting tool calling, instruction following, agentic tasks and Indian languages. The main repository contains NVFP4 weights; a separate [BF16 checkpoint](https://huggingface.co/sarvamai/sarvam-105b-1711-bf16) is available. These are deployment choices, not two different model generations.

Here is the migration sequence I would use. It is a proposed evaluation plan, not a report of running this model myself.

## Keep the old endpoint while testing the new one

Start with a separate candidate endpoint and an explicit checkpoint revision. Give it a different internal name from the production model, even if both eventually sit behind the same public route. That makes logs useful: an unexpected tool call should be traceable to a particular deployment rather than an ambiguous model alias.

Write down the existing container version, tokenizer, chat template, parser settings, quantization and request defaults before changing anything. Save a small set of representative requests with sensitive content removed. Include the exact tool schemas and the application response that followed, not just the user's question.

My first test would be a short conversation that calls a harmless read-only tool, receives its result and answers the user. Then I would try a request that should not call a tool. Both paths matter. An assistant that produces attractive text but sends malformed arguments is not a successful migration.

## Preserve the reasoning and tool interface

Sarvam documents a vLLM 0.33.0 setup using `glm45` for both tool-call and reasoning parsing, with automatic tool choice enabled. Thinking is on by default; its recommended temperatures are 1.0 with thinking and 0.6 without it. The documented per-request switch is `chat_template_kwargs.enable_thinking`.

Treat those settings as a starting configuration to validate, rather than copying only the checkpoint name into an older deployment. Test the response fields your application actually consumes. Check that the user-facing answer is displayed correctly and that intermediate reasoning is not accidentally concatenated into it.

For tool calls, inspect the parsed name and arguments before any execution. I would keep the candidate connected to stubbed tools initially, especially for workflows that write records or send messages. An evaluation should reveal the model's intended action without letting a formatting regression become a real business action.

## Separate capacity testing from quality testing

The card provides serving examples with and without DSpark speculative decoding. Its throughput results use specified hardware and workloads: the B200 example uses 128k context, while the RTX PRO 6000 Blackwell example uses 50k. Those are not interchangeable capacity promises.

First establish a working baseline without changing every optimization at once. Then compare the candidate configuration against the same request set. Record time to first token, completed-request latency, queue time and failure rate at the concurrency your application actually sees.

Include long conversations and tool-heavy turns in that set. A successful short greeting is a useful smoke test, but a poor basis for deciding how many users a server can support. Keep output limits and timeout policies identical during the comparison so you can explain the result.

## Make the rollout decision in application terms

For an Indian-language support assistant, I would review natural responses, mixed-language messages and tool results in the languages customers use. Ask reviewers to judge whether the task was completed correctly, not simply whether the answer sounded fluent.

Expand traffic only after the candidate passes those checks. Keep a route back to the previous endpoint and a record of the tested configuration. That gives you a reversible upgrade, instead of a weight swap whose behavior you have to reconstruct after users notice a problem.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams turn model releases and deployment details into technical stories developers can use.

[Book a 30-minute call](https://cal.com/paras-madan/30min).
