Your Approval Dialog Is Not a Security Control

If humans say yes to almost everything, human-in-the-loop is theater. Here is how to build an oversight layer that actually stops something.

Anthropic published a number that should bother anyone shipping agents: users accept 93% of Claude Code's manual permission prompts (Anthropic Engineering). Not 93% of the dangerous ones. 93% of all of them.

That is the whole problem with how most teams implement human-in-the-loop. You wire a confirmation dialog in front of every tool call, you ship it, you write "human oversight" in the architecture doc, and three weeks later your users are clicking yes faster than the renderer can paint the button. The dialog is still there. The oversight is gone.

A click-through rate that high means the prompt carries almost no signal. If a reviewer approves nearly everything, the gate is not filtering. It is logging, with extra latency.

The prompt is the wrong layer

Interrupting on every action trains people to stop reading. Interrupting on nothing gives the agent a blank cheque. The fix is not a better dialog. The fix is deciding, before runtime, which actions a human is even allowed to approve.

Three things have to be true at once for an approval to mean anything.

The action has to be rare enough that the reviewer has not gone numb. The reviewer has to see the actual call, not a paraphrase of it. And the set of actions that no approval can unlock has to be enforced somewhere the model cannot reach.

That last point is where most implementations fail. If your only control is a prompt the agent triggers, the agent's context is part of your security boundary. Context is attacker-influenced. Boundaries should not be.

Classifiers shift the problem, they do not solve it

The obvious scaling move is to replace the human with a model that reviews tool calls. Anthropic's own published evaluation of that approach is worth reading carefully. On real internal traffic the full pipeline got the false positive rate down to 0.4%, which is excellent. On a set of 52 real overeager actions, the false negative rate was 17%.

Roughly one in six genuinely risky actions got through. And the second stage that cut false positives actually made recall worse.

That is a reasonable tradeoff for low-stakes work. It is not a control you put in front of a production database drop or a wire transfer. NeuralTrust's breakdown of Claude Code's security model walks through where each built-in layer stops covering you, and the pattern generalizes to any agent framework. Classifiers are a good way to reduce noise above a hard floor. They are a bad floor.

Put the limit outside the model

The enforceable version of human-in-the-loop is a policy layer between the agent and every tool, MCP server, and downstream API it can reach. The agent asks. The layer decides. The model never sees the decision logic and cannot argue with it.

Concretely, something like this, evaluated before the call is dispatched:

TIERS = {
    "read":      "auto",      # list_files, search, get_record
    "write":     "notify",    # reversible, undo window, no blocking prompt
    "external":  "approve",   # send_email, post_webhook, charge_card
    "privileged":"two_party", # grant_role, rotate_key, delete_bucket
}

BLOCKED = {"disable_audit_log", "drop_database", "modify_iam_policy"}

def authorize(agent_id, tool, args):
    if tool in BLOCKED:
        raise PolicyDenied(f"{tool} has no approval path")

    tier = TIERS[classify(tool, args)]

    if tier == "auto":
        return Allow()
    if tier == "notify":
        return Allow(undo=True, notify=agent_owner(agent_id))

    # Deny by default. A timed-out approval is a denial, never a pass.
    decision = request_approval(
        agent=agent_id, tool=tool, args=args,
        reviewers=2 if tier == "two_party" else 1,
        timeout=300, on_timeout=Deny,
    )
    return decision

Two details in there do most of the work.

BLOCKED is a set with no approval path at all. Not "requires senior approval." No path. If an action can destroy evidence or escalate privilege, a tired human at 23:00 should not be able to unlock it from a chat window.

And on_timeout=Deny. An approval request that nobody answers has to fail closed. Fail-open timeouts are how a careful design turns into an unattended agent with full rights at the weekend.

The same argument applies to credentials. Scope the token so the dangerous call returns a 403 even if every other layer is bypassed. A gateway sitting in front of agent traffic is the practical place to put identity, allowlists, argument validation, and the approval hook, because it is the one component the agent cannot route around.

Tiering beats blanket approval

Four tiers is usually enough. Read-only runs unattended. Reversible writes run with a notification and an undo window. Anything that leaves your perimeter or moves money blocks on approval. Anything destructive or privilege-changing needs a named approver and a second reviewer.

NeuralTrust's full guide to human-in-the-loop AI agents goes deeper on the oversight models behind this, including the difference between a human in the loop approving each action and a human on the loop watching a fleet and intervening. The distinction matters once you have more agents than reviewers, which happens faster than teams expect.

The approval UI is attack surface

This is the part that gets skipped.

Your approval screen renders content that an attacker may control. Indirect prompt injection can shape the summary a reviewer reads, so the human approves one thing and the system executes another. Render the resolved call. Raw tool name, raw arguments, resolved destination, after templating. Never a model-written description of what it is about to do.

Approval messages also leak. Pushing a payload into a Slack channel so five people can see it moves regulated data into a system that may not be in scope for it.

And per-action review has a structural blind spot. Ten individually unremarkable reads add up to an export. If your gate only evaluates single calls, chain-level behavior is invisible to it. Rate limits per run and per agent per day catch what a per-call policy cannot. Agent Security has a useful catalogue of these failure modes, including memory poisoning and autonomous drift, which both bypass action-level review by design.

Measure the gate, not the intent

Instrument three numbers from day one.

Approval rate per tier. If a tier sits near 100%, it is not a gate and you should demote it to notify-with-undo. Time to decision. If reviewers answer in under two seconds, they are not reading. Override and retry rate. Agents that get denied and immediately rephrase the same request are a signal worth alerting on.

Log every decision with the agent identity, the exact arguments, the reviewer, and the policy version in force. Without the policy version you cannot reconstruct why something was allowed six months ago, which is the first question any auditor asks.

One compliance note

Article 14 of the EU AI Act requires high-risk systems to be designed so humans can oversee them, including the ability to intervene and stop operation. The deadline moved. Under Regulation (EU) 2026/1744, the Digital Omnibus that entered into force on 27 July 2026, Annex III high-risk obligations now apply from 2 December 2027 instead of August 2026 (Gibson Dunn).

Whether your specific system falls in Annex III is a question for counsel, not for a blog post. But a stop button that only works when the agent chooses to surface it does not satisfy a meaningful reading of Article 14.

The short version

Oversight is not a dialog. It is a tiered policy, enforced outside the model, with a blocked set that has no approval path, denial on timeout, scoped credentials underneath, and metrics that tell you when a tier has stopped working.

If your approval rate is 93%, you do not have human-in-the-loop. You have a very expensive confirm button.