Controls to add before an AI agent takes action

An agent that can call tools or move money fails in a different place than a chatbot that only replies. I assume most builders start with software permissions, an API key, and a policy sitting in the prompt. I would not stop there. Four episodes each hand you one decision: how you patch, how you bound a tool, where governance lives, and when hardware has to sign the action.

What changes if an attacker needs no human?

In The First Fully Autonomous AI Attack is Coming, Kris Lovejoy, Global Head of Strategy at Kyndryl, predicts a fully autonomous attack that takes an enterprise network down with no human driving it, inside 18 months. Her reasoning runs through attackers chaining low-risk vulnerabilities. That changes the patching math. A bug you scored as low risk is still a step in a chain an agent can walk without a person at the keyboard.

She separates individual productivity use from mission-critical business transformation. The same episode notes that 62% of enterprise AI initiatives are stuck in pilots, and that agentic AI still needs a real control plane. I would read those together. A productivity assistant can live in a pilot. A workflow that can take a network down should not borrow that pilot's missing controls. I would not point an autonomous agent at a mission-critical system while that control plane is still undefined.

Practical move: when you triage, ask whether a low-risk finding becomes useful once something can chain it with others. Patch and segment for that case, even when each item looks small on its own.

How do you keep a bad prompt from using a strong tool?

Malte Ubl, CTO of Vercel, describes agents as a new type of software for solving flexible tasks. The security lesson he says people get wrong most often is prompt injection. Teams treat it like SQL injection and filter what goes in. He argues for constraining what the tool can do.

My inference is that flexible tasks are why the tool list grows. The agent is supposed to pick a path. That is also why a filter on the inbound text is a weak control. Text that gets past the filter still meets every function you exposed. I would write the constraint on the tool before spending time on the filter.

This sketch is mine, for illustration. It is not a vendor SDK, and it is not a policy Malte spelled out.

type Effect = "read" | "draft" | "send" | "pay";

const allowed: Record<string, Effect[]> = {
  inbox: ["read", "draft"],
  billing: ["read"],
};

function permit(tool: string, effect: Effect): boolean {
  return (allowed[tool] ?? []).includes(effect);
}

If permit returns false, the call does not run. The prompt can ask for pay. The map does not grant it. Widen the map only when you mean to.

Where do you put rules the agent cannot edit?

Tyler Akidau, CTO of Redpanda and author of the O'Reilly Streaming Systems book, argues that enterprises are shipping agents into production without the governance layer they need. He lays out four pillars of agent HR: identity, authorization, observability, and accountability. Guardrails written where the agent can see them are not guardrails. Task-scoped identity is the foundation the rest sits on. Inline enforcement via CLAUDE.md fails the moment the stakes get real.

I would build in that order. Create an identity scoped to the task, authorize against that identity, record what the task did, and name a person accountable for the outcome. A markdown file the agent loads as instructions can help in a prototype. It is still inside the agent's view. When the task can spend money, change data, or call production tools, move the rule to a checker the agent cannot rewrite.

Observability is one of the four pillars, and it is the one a working demo makes easy to skip. If you cannot see what the agent did, accountability has nothing solid under it. Log the task from a system the agent does not control.

When should hardware sit between intent and payment?

Charles Guillemet, CTO of Ledger, runs the offensive security lab that breaks Ledger's own products before attackers can. He argues you cannot secure an agent with software permissions and API keys once it is moving real money. The control he makes the case for is a hardware-enforced signature between intent and execution. Hardware has to be in the loop.

I would split agents into two piles. One pile drafts, searches, and opens pull requests. Software limits, task identity, and narrow tools can be enough to start. The other pile pays, transfers, or signs something with monetary value. For that pile I would not accept an API key and a trust scope as the last check. The agent can propose an intent. A hardware step has to sign before execution. Charles's lab exists to break Ledger's products before attackers do. I would take the same stance toward software permissions: assume a fooled agent can use whatever you handed it in software.

Checklist

• Triage vulnerabilities as links in a chain. Kris Lovejoy's prediction is an autonomous attack inside 18 months, with no human driving it, built by stacking low-risk bugs.
• Keep productivity pilots and mission-critical systems apart. A pilot is not a control plane, and 62% of enterprise AI initiatives are still stuck in pilots.
• Constrain each tool to an allowlist of effects. Filtering the prompt leaves a powerful tool in place.
• Issue task-scoped identity first. Enforce authorization, observability, and accountability outside anything the agent can read and edit, including CLAUDE.md-style files.
• If the agent moves real money, require a hardware-enforced signature between intent and execution. Software permissions and API keys do not cover that case.

More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.