Gemini's Tool Retrieval: Stop Paying for Tools Your Agent Never Calls
Every tool you give an agent costs tokens on every single request — whether the agent uses it or not. Google's answer, launched November 19, 2026 in preview in the Gemini Interactions API, is Tool Retrieval: keep the full tool library on standby and load a tool's parameter schema only when the current task actually needs it.
The problem it solves
When an agent connects to dozens or even hundreds of tools, sending every full parameter schema on every request adds up fast. Google names three costs: inflated input token counts, added latency, and lower tool-selection accuracy as the candidate list grows. The bigger the library, the worse each of these gets — and the more of the library sits idle on any given turn.
How it works
Turning it on takes two changes to your request:
- Add a
tool_searchtool to the request. - Set
defer_loading: trueon the tools you want to hold back.
The model still sees the name and description of every deferred tool, so it knows what is available. The parameter schemas stay out of context. When the model needs a tool, it asks for it by name, the API loads the full schema, and the model calls the function — all inside the same request.
Google's example is a customer-service agent: a customer reports order A-1042 arrived damaged and asks for a 40-dollar refund. Inside one API call, the model searches for issue_refund, the API loads its schema, and the model emits the function call. All three steps — tool_search_call, tool_search_result, and function_call — appear in interaction.steps, so you can inspect which tools the model considered when a call goes wrong. That trace is the debugging story: bad tool selection usually traces back to a vague name or description, and now you can see exactly what the model saw.
There is no index to build or host. Gemini runs the search on the server, so the only code changes are the new tool and a flag on the tools you defer. Google's own advice is to keep your most-used tools loaded up front and defer the long tail — the tools that are useful once in a while but not on most turns.
It plays well with the rest of the stack
Two integrations are worth noting:
Prefix caching. Deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn. If you are already paying for prefix caching, deferred loading does not fight it — it makes the cache more stable.
Remote MCP servers. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand. You can connect a large MCP server without listing its tools one by one, which removes one of the main reasons people avoid wiring big servers into agents.
Client-side retrieval, for your own registry
If you maintain your own tool registry, embedding index, or per-user permission filter, declare tool_search with "execution": "client" and define your own search parameters. Your app runs the lookup, which adds one round trip per search. The trade-off buys you two things server-side retrieval cannot: tools do not need to be declared upfront, so you can expose new tools mid-conversation when a user connects an integration, and you control ranking and filtering yourself — including per-user permissions, which matters the moment different users are allowed different tools.
What it costs
Tool Retrieval has no separate charge. Input and output tokens are billed at the standard rates for the model you call. Turning on tool_search adds a small, fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation, exactly like a tool declared up front. You save tokens when the schemas you never load outweigh the ones you do — which, for a large library, is the usual case.
That "usual case" is doing a lot of work, though. The announcement includes no measured savings figures, no latency numbers, and no tool-selection accuracy benchmarks. Google's own guidance is to measure it against your own traffic. The honest way to read that: if your agent calls most of its tools on most turns, the fixed cost of tool_search plus the schemas you do load may not beat declaring everything upfront. If your library is a long tail — twenty tools that each fire once a day — the math is clearly in your favor.
Availability
Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, in the Interactions API. It is in preview, so the API surface can change before general availability — pin your model version and keep an eye on the interaction.steps shape if you build tooling around it.
The bigger picture
Deferred tool loading is quietly becoming the default shape of agent APIs: declare the catalog, load the details on demand. The interesting shift is not the token savings alone. It is that the boundary between "tools the agent has" and "tools the agent can see" is becoming a first-class design decision. Name and description become the retrieval surface, and the quality of those two fields decides whether the model finds the right tool at all. If you adopt Tool Retrieval, the highest-leverage work is probably not the flag — it is rewriting your tool descriptions so a search can find them.
Source: Google, "Gemini API tool retrieval for efficient agent scaling," November 19, 2026 — https://blog.google/innovation-and-ai/technology/developers-tools/gemini-api-tool-retrieval/