PARAS MADAN
All writing

Latest in AI

Gemini API Tool Retrieval: Setup, Token Costs, and 400 Errors

Set up Tool Retrieval in the Gemini Interactions API, choose which functions to defer, and measure whether it lowers cost per completed task.

A support assistant might have 80 functions but use only two for most tickets. That does not mean it needs to read 80 parameter schemas every time someone asks where their package is. Google's November 19 Tool Retrieval preview gives teams a way to load the less common schemas when needed. It does not make search itself free of token costs, and it changes what your app must preserve across turns.

Check the endpoint before changing the tool list

Tool Retrieval is a preview on the v1beta Interactions API, not on generateContent or stable v1. The feature guide lists supported models, including Gemini 4 Hydrogen, Carbon, Argon, and Neon, plus Gemini 3.8 and 3.6 Flash. Use a listed model your project already has access to.

If you're moving from generateContent, first get the existing static tool list working on Interactions. Use your usual read-only function and reproduce a full turn, including the application returning its result. The Interactions API overview explains what a saved interaction carries forward. Moving endpoints and adding retrieval in the same change makes it hard to tell where a regression started.

Keep the default tools loaded

Start with a tool that customers need every day, such as checking order status. Keep that function available up front. Add {"type": "tool_search"} to the tools array and set defer_loading: true on a less common function, such as looking up a regional exchange rule. Send the full parameter schema in the declaration; deferring it changes when the model sees it, not what your app supplies.

Google handles the lookup inside the request and surfaces search steps before a custom function_call. Your application still runs the custom function and returns a function_result. Test an order-status question first, then an exchange question, then a question that needs both. Inspect the steps to see whether a search happened and which function the model ended up calling. Use two exchange functions with similar names to check whether their descriptions actually make the difference clear.

Pick client-side search if the tool list changes by user

A static list is the easiest baseline, especially if the same handful of tools handles most work. Server-side search suits a larger set that your app already knows. For a registry that changes by account, Google's launch example shows execution: "client". Your code looks up matching declarations and sends them back in another request. Use one retrieval mode per request.

Treat the search result as a list of candidates, not permission. A support assistant should only see functions that the current user may use, and the system that actually executes the function still has to check identity and access. For example, if a support contractor loses access halfway through a session, a schema loaded earlier must not be enough to fetch a customer record.

Fix follow-up turns before rollout

With previous_interaction_id, send the tool array again on follow-ups, including the call that returns a function result. If you're storing history yourself, keep the returned search steps in order and preserve their signatures. Missing search context or signatures can produce a 400.

A useful regression test sends a function result round trip through the same storage and serialization path you use in production. If calls work in a scratch script but fail after the app reloads a conversation, compare the stored steps against the original response before changing the prompt. Check for a dropped step type, signature, or missing tool_search in the next request.

Measure the whole ticket, not the first prompt

There is no separate Tool Retrieval fee, but server-side search does another model pass within the request. In the pricing and token usage notes, Google says this input is reported in total_tool_use_tokens. Loaded schemas can stay in the conversation, so savings on the first call are not a reliable estimate of the whole session. Inspect cached tokens too: Google's caching guide lists a 4,096-token minimum for Gemini 4 models. A smaller prompt may give up a cache hit the larger prompt would have received.

Replay the same tickets with one model, the same tools, and both configurations. Include common requests, long conversations, and requests that need several lookups. Compare cost per successfully completed ticket, time to the final answer, and wrong-tool calls. If retrieval wins on rare workflows but slows down everyday ones, keep it on the rare path. You don't have to migrate the whole assistant to save on the part with the biggest catalog.

About Paras Madan

I'm Paras Madan. I write about AI products and how teams use them.

If you're building an AI product and need clearer technical writing or a better way to explain what you shipped, I can help.

Book a 30-minute call.