Check each agent action after you grant the tool

Granting an agent a tool does not approve every action that tool can perform. Check the action and the class of target before the call, and monitor overly consumptive token use so you can quarantine those agents. Build evals on your own tasks before you route a prompt to a cheaper or a more expensive model, because a public benchmark score does not establish how the model behaves in your environment.

Jeetu Patel, President and Chief Product Officer at Cisco, made that case on Chain of Thought in August 2026.

How do you tell a tool grant from an allowed action?

A tool grant answers whether the agent may use the integration. An action check answers what it may do, and to whom. Patel's inbox example keeps those apart: making a dinner reservation can be in bounds, while contacting a board of directors is a different action on the same connection.

You can't just do access control on the agents. You have to do action control, which means, Conor, if you wanted to grant an agent rights to email, to your email inbox, You don't want to grant it full carte blanche rights to your email inbox. You want it to grant rights to it saying, hey, you know what? You can go out and make reservations for me for a restaurant for dinner. But what you can't do is reach out to my board of directors and start randomly texting them.

Jeetu Patel, President and Chief Product Officer at Cisco, on Chain of Thought ep 67

Parenting Your AI Agents for B… - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts

Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · August 12 · 46m

podcasts.apple.com

Store the decision as a key of tool, action, and target class. ("inbox", "reserve", "restaurant") and ("inbox", "message", "board") need different answers. Intercept any triple you have not already approved. Enforce that allow list outside the model, and pair it with runtime intent checks. Agents are too smart for static rules and bypass them quickly, in Patel's account. Boundary conditions have to be created at runtime, from whether current behavior is drifting from the original intent, with an intercept when it drifts.

How do you supervise actions and token use?

Intercept the call when the agent crosses a boundary you set, and quarantine it when token consumption runs away. Patel places that next to infrastructure monitoring: resiliency of the system, guardrail crossings, and token use.

And number three, you have to make sure that you're monitoring tokenomics, that these agents aren't being overly consumptive in their patterns of usage of tokens.

Jeetu Patel, President and Chief Product Officer at Cisco, on Chain of Thought ep 67

Quarantine is how he would put an overly consumptive agent on ice. Cisco started by giving people broad access to tokens so they could get familiar with them, then moved to rationing tokens the way headcount is rationed. A team gets a certain number of tokens and an output to deliver from them. The next question, as he put it, is how to build the architecture so it gets 20, 30, 40, 50, 60 percent more lift in tokens generated for the same cost.

A minimal sketch to adapt, not a drop-in library.

def check_action(agent, tool, action, target_class, tokens_needed):
    if (
        isinstance(tokens_needed, bool)
        or not isinstance(tokens_needed, int)
        or tokens_needed < 0
    ):
        raise ValueError("tokens_needed must be a nonnegative integer")
    if agent.quarantined:
        return "blocked"
    if agent.tokens_used + tokens_needed > agent.token_budget:
        agent.quarantined = True
        return "quarantine"
    key = (tool, action, target_class)
    if key not in agent.allowed_actions:
        return "intercept"
    agent.tokens_used += tokens_needed
    return "allow"

allowed_actions holds the triples you have approved. Crossing token_budget sets the quarantine flag so later calls stop until a person clears it. Compare the live action with the task intent as the agent works, and intercept when they diverge. A rule frozen at setup is what Patel says agents route around.

Why build your own evals before routing models?

A high score on a public eval does not mean the model will do well on your use case. It means the model did well on the behaviors that benchmark measured. Build evals around your tasks first, then use those results to send a prompt to a cheaper model or a more expensive one.

And by the way, your benchmarks and evals don't tell you the full story, which is the other thing that most people don't realize. It's like, okay, if a model does well on one eval, that doesn't mean the model is going to do well in your environment. It just means it does well in that eval for which that, for that, where that benchmark was evaluating a very particular set of behaviors of the model where that model performed well.

Jeetu Patel, President and Chief Product Officer at Cisco, on Chain of Thought ep 67

Cost, in his framing, includes the dollar price of the token and the wattage it consumes, because you can have the money and still lack the power. He also said 60 percent of the total compute capacity that's available is going toward inferencing. The trade-off to write down per task is intelligence, cost, and control. On a task where control matters more, such as sensitive data, he said the last 5 percent of intelligence might not matter as much.

Leave a person in the approval path until that use case has a track record you trust. Patel's version: approve everything until the models are fine-tuned enough to your environment and you have seen enough of their decisions statistically to feel comfortable, and only then consider taking the human out of the loop. Where that point falls differs by use case and organization, and between simple and more complicated tasks.

FAQ

Does an inbox connection let the agent send any message?
The connection is the tool grant. Patel splits dinner reservations from contacting a board of directors, and those need separate action checks on the same inbox.

Will a public benchmark tell me which model to route to?
It reports how the model did on that eval's behaviors. Patel says that result does not establish performance in your environment. Build evals for your use cases, then route.

What should happen if an agent burns tokens or leaves its bounds?
Intercept the action that crosses the boundary. If token use is overly consumptive, quarantine the agent so it can be put on ice.

Takeaway

• Key each sensitive call by tool, action, and target class, and intercept keys you have not approved.
• Set a token budget per agent and quarantine the agent when that budget is exceeded.
• Recompute boundary checks at runtime against the original intent, alongside authorization enforced outside the model.
• Build evals on your own use cases before you route prompts between cheaper and more expensive models.
• Keep a human on approvals until that use case has a track record you trust.

The full conversation, with the transcript, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.