Lazy-Load Tool Schemas. Keep Authorization Eager.
A large tool catalog creates a real context problem. Loading every parameter schema into every request makes the agent pay for tools it will never use. The obvious fix is to load schemas on demand. The dangerous mistake
A large tool catalog creates a real context problem. Loading every parameter schema into every request makes the agent pay for tools it will never use.
The obvious fix is to load schemas on demand. The dangerous mistake is to treat that optimization as a permission system.
A tool can be discoverable without being usable. A schema can be loaded without authorizing the requested action. Those are separate states, and production agent systems need to keep them separate.
Discovery is not authority
A useful mental model has three surfaces:
- Catalog metadata: the tool name and a short description that can be searched.
- The full schema: parameters, constraints, and the shape of the request for one selected tool.
- Execution authority: whether this user, agent, target, and action are allowed right now.
The first two are about selection and context management. The third is a decision at the side-effect boundary.
Google's Tool Retrieval preview makes the separation visible in its flow: retrieval, schema loading, and function calling appear as distinct steps. MCP client guidance describes a similar progressive-discovery patternβcatalog, search or filter, inspect the schema, then executeβwhile also treating access-control filtering as a host or application concern.
That separation matters even when every step is handled by the same model. A description that helps the model find delete_customer must not become evidence that the current request may delete a customer.
A practical pipeline
For a tool-using agent, the flow should look something like this:
| Stage | What it does | What it must not decide |
|---|---|---|
| Catalog | Exposes searchable names and descriptions | Whether an action is permitted |
| Search/filter | Narrows the candidate set | Whether the selected tool is safe to call |
| Schema load | Supplies parameters for the selected tool | Whether the caller has current authority |
| Authorization check | Validates identity, scope, target, and freshness | The tool's parameter shape |
| Execution | Performs the approved call | A new or changed action without another check |
| Receipt | Records the decision and result | That a successful call proves future permission |
The important detail is where the authorization check sits: immediately before execution, after the concrete tool and arguments are known.
A permission decision made during catalog search can go stale. The user's role may have changed. A resource may have moved to a different tenant. The agent may have turned a harmless read request into a write request while filling in parameters. The call boundary is where those differences become concrete.
A minimal policy object might therefore carry the request through an explicit gate:
const candidate = await searchCatalog(query)
const schema = await loadSchema(candidate.toolName)
const args = await planArguments(schema, task)
const decision = await authorize({
identity: session.identity,
tool: candidate.toolName,
args,
target: resolveTarget(args),
scope: session.scope,
})
if (!decision.allowed) {
return recordDeniedCall(decision)
}
return executeAndRecord(candidate.toolName, args, decision)
This is a proposed shape, not a tested implementation. Its value is the boundary: discovery produces a candidate, while authorization produces a current decision tied to the actual call.
Why schema deferral is worth considering
The context cost is not theoretical. One DEV.to experiment reported 24,522 tokens for 200 synthetic MCP tools and 61,302 for 500 before the agent did useful work. The same author found that a real file server used more tokens per tool than the synthetic set.
Those numbers are measurements from one setup, not a universal budget or a benchmark reproduced here. They still point to the right engineering question: how much of the request is being spent describing tools that will not be selected?
Schema deferral can reduce that menu overhead by keeping lightweight metadata available and loading full definitions only for plausible candidates. It also makes the retrieval decision observable. If the trace distinguishes search, schema load, and function call, operators can see whether an agent is failing because it cannot find a tool, cannot form valid arguments, or cannot pass authorization.
Do not collapse all costs into βtool overhead,β though. Tool definitions, retrieved results, conversation history, and an extra retrieval turn are different surfaces. A progressive-discovery design can reduce one while increasing another through additional latency or model calls.
For a small, stable catalog, eager loading may still be the simpler and faster choice. Switch to progressive discovery when measurements show that definitions consume a meaningful share of context, not because lazy loading sounds more sophisticated.
Failure modes to test explicitly
A discovery-first design needs negative tests, not just a happy path.
- Revocation after discovery: the user can search for a tool, then lose the scope required to call it.
- Target drift: the selected resource changes between retrieval and execution.
- Description overreach: a broad description makes a tool appear suitable for actions its policy does not permit.
- Visible but unusable tools: catalog visibility is global, while execution may depend on tenant, role, environment, or resource state.
- Retrieval inflation: the search step adds enough latency or context to erase the savings from deferred schemas.
- Ambiguous denial: the system records only βtool failed,β leaving reviewers unable to distinguish discovery failure, policy denial, and execution failure.
At minimum, log the discovered candidate, the concrete requested action, the authorization decision, and the execution result as separate events. Test revocation between discovery and execution. Make denial explicit and auditable rather than returning a generic tool error.
The adoption rule
Keep the catalog simple until it becomes expensive. When it does, make schema loading progressiveβbut keep authorization eager.
The useful boundary is not βthe model may see this tool.β It is βthis exact identity may perform this exact action against this exact target under the current policy.β Tool retrieval helps answer the first question. It does not answer the second.
That distinction is small in a diagram and decisive in production. Optimize context at discovery time. Recompute authority at execution time. Record both decisions so the next failure is diagnosable.
Source notes
- Introducing Tool Retrieval in the Gemini API β official preview describing deferred schemas and separate retrieval, schema-load, and function-call steps.
- I Gave My Agent 200 MCP Tools β It Cost 24,522 Tokens Before It Did Any Work β source-reported measurements with synthetic and real-server caveats.
- Client Best Practices β progressive discovery, thresholds, and host-side access-control guidance.
- Scalekit on Product Hunt β supporting market signal about tool gateways, credentials, per-agent toolsets, and call logs; maker claims only.
- Permission-after-discovery discussion β supporting design prompt about separate discovery, requested-action, and verified-outcome records.
- November HN hiring thread β supporting, self-reported community example of OAuth and approval around a broad MCP server.
The workflow and code are proposals, not hands-on test results. The reported token figures are retained as the source author's measurements, not as benchmarks from this article.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.