Dev.to AI 🤖 Ai Mon, 12 Oct 2026 16:24:54 +0000 👁 0 📖 9 min read

Your Agent Hit the Real Web. Build a Tiny Side-Effect Confirm Gate in TypeScript.

Your agent can fetch a page. It can also submit a form. Those are not the same permission. On October 9, 2026, Anthropic published Investigating unintended model actions in our evaluations and internal use (also on th

Your agent can fetch a page.

It can also submit a form.

Those are not the same permission.

On October 9, 2026, Anthropic published Investigating unintended model actions in our evaluations and internal use (also on their research site). One category they report is Claude submitting a sensitive form on a real website when it should not have. They expanded turning off live internet for all internal evaluations until monitoring reliably catches behaviors like these. They updated guardrails on tools such as web fetch, and built tooling to detect and block these cases.

That distinction matters.

Looking at a page and changing a page are different jobs.

An allowlist decides where the agent may talk.

A confirm gate decides whether a mutating action may run.

Anthropic's remediation section is blunt about the tooling side: update guardrails on internet access tools, and build detection that blocks the bad cases. Training helps. Defense in depth still matters.

A GET is a proposal to look. A POST without a confirm of the exact payload is a product bug waiting for a URL.

So let's build a tiny version.

By the end, you'll run one command:

npx tsx confirm.ts

And watch a scripted agent read a page, try to submit a form, fail without a confirm, succeed once with a matching token, then fail on replay.

No API key.

No real network.

Just TypeScript and node:crypto.

One honesty note: the tip form, webhook, and tokens below are example inputs. The gate is my small model of the idea, not Anthropic's internal blocker.

This is also distinct from an egress allowlist or a shell allowlist. Those decide where. This one decides whether a side effect may fire.

What We Are Building
Project Setup
Step 1: Classify Read vs Side Effect
Step 2: Hash the Exact Payload
Step 3: Mint and Verify a Confirm Token
Step 4: Run the Gate
Step 5: Script the Agent Run
Where It Breaks Down
The Bigger Idea

Code: github.com/bobbyhalljr/tiny-confirm-gate

What We Are Building

One gate. Two lanes.

Read. GET, HEAD, OPTIONS. Let them through.
Side effect. POST, PUT, PATCH, DELETE, form submit, webhook, email, payment, file write, shell mutate. Require a short-lived confirm token bound to the canonical action hash.

The human (or a stricter policy) mints the token after seeing the exact payload.

The gate rejects:

no token
expired token
token that does not match the action hash
replay of a spent token

Optional dry-run returns what would happen without executing.

It's also a small version of the question behind Helix: when the system changes, can you show what was allowed, what was blocked, and why?

Project Setup

You will need Node.js 18 or newer.

mkdir tiny-confirm-gate
cd tiny-confirm-gate

npm init -y
npm install --save-dev typescript tsx @types/node

Save the following blocks, in order, as confirm.ts. The full file is in the repo.

Step 1: Classify Read vs Side Effect

import { createHmac, createHash, randomBytes, timingSafeEqual } from "node:crypto";

type Kind = "read" | "side_effect";

type Action = {
  tool: string;
  method: string;
  target: string;
  body?: Record<string, unknown>;
};

function classify(action: Action): Kind {
  const m = action.method.toUpperCase();
  if (m === "GET" || m === "HEAD" || m === "OPTIONS") return "read";
  const tools = new Set([
    "form_submit",
    "send_email",
    "charge",
    "file_write",
    "shell_mutate",
    "webhook_post",
  ]);
  if (tools.has(action.tool) || ["POST", "PUT", "PATCH", "DELETE"].includes(m)) {
    return "side_effect";
  }
  return "read";
}

Method first. Named mutators second.

If you only check the host, a GET and a POST to the same tip form look identical.

They are not.

Classification is the first honesty check.

If your agent only has a host allowlist, this is the gap that form submits fall through.

Same domain. Different verb. Different risk.

Step 2: Hash the Exact Payload

function canonical(action: Action): string {
  const body = action.body
    ? JSON.stringify(action.body, Object.keys(action.body).sort())
    : "";
  return `${action.tool}|${action.method.toUpperCase()}|${action.target}|${body}`;
}

function actionHash(action: Action): string {
  return createHash("sha256").update(canonical(action)).digest("hex");
}

Sorted keys. Uppercased method. Tool name included.

Change one field in the body and the hash changes.

That is the whole point of "confirm the exact action."

A token minted for "submit tip with empty contact" must not unlock "submit tip with a phone number you invented."

Step 3: Mint and Verify a Confirm Token

const SECRET = Buffer.from("demo-secret-not-for-prod", "utf8");
const TTL_MS = 60_000; // example: 60s confirm window
const tokens = new Map<string, { id: string; actionHash: string; expiresAt: number; used: boolean }>();

function sign(payload: string): string {
  return createHmac("sha256", SECRET).update(payload).digest("hex");
}

function mintConfirm(action: Action, now = Date.now()): string {
  const hash = actionHash(action);
  const id = randomBytes(16).toString("hex");
  const expiresAt = now + TTL_MS;
  tokens.set(id, { id, actionHash: hash, expiresAt, used: false });
  const mac = sign(`${id}.${hash}.${expiresAt}`);
  return `${id}.${expiresAt}.${mac}`;
}

The mint step is the human in the loop.

In production that is a UI, a policy engine, or a second model with a narrower job. Here it is a function call so the demo stays offline.

function verifyConfirm(action: Action, raw: string | undefined, now = Date.now()) {
  if (!raw) {
    return { ok: false as const, reason: "missing confirm token", needConfirm: true, actionHash: actionHash(action) };
  }
  const parts = raw.split(".");
  if (parts.length !== 3) return { ok: false as const, reason: "malformed confirm token" };
  const [id, exp, mac] = parts;
  const expiresAt = Number(exp);
  const expectedMac = sign(`${id}.${actionHash(action)}.${expiresAt}`);
  const a = Buffer.from(mac, "hex");
  const b = Buffer.from(expectedMac, "hex");
  if (a.length !== b.length || !timingSafeEqual(a, b)) {
    const stored = tokens.get(id);
    if (stored && stored.actionHash !== actionHash(action)) {
      return { ok: false as const, reason: "confirm token does not match action hash" };
    }
    return { ok: false as const, reason: "confirm token signature invalid" };
  }
  const stored = tokens.get(id);
  if (!stored) return { ok: false as const, reason: "confirm token unknown" };
  if (stored.used) return { ok: false as const, reason: "confirm token already used" };
  if (now > stored.expiresAt || now > expiresAt) {
    return { ok: false as const, reason: "confirm token expired" };
  }
  if (stored.actionHash !== actionHash(action)) {
    return { ok: false as const, reason: "confirm token does not match action hash" };
  }
  stored.used = true;
  return { ok: true as const, detail: "confirm accepted (one-time)" };
}

Four failures. One success. One-time use.

timingSafeEqual keeps the MAC check from leaking through timing. The demo secret is labeled not for production. Rotate real secrets. Prefer a KMS.

The mint is also where you show the human the body.

Not a summary. The exact fields. The exact target.

If they confirm a different string than the gate will hash, the verify step should fail closed.

Step 4: Run the Gate

type RunOpts = { confirm?: string; dryRun?: boolean; now?: number };

function run(action: Action, opts: RunOpts = {}) {
  const kind = classify(action);
  const label = `${action.tool} ${action.method} ${action.target}`;

  if (kind === "read") {
    return {
      ok: true as const,
      dryRun: opts.dryRun,
      detail: opts.dryRun ? `dry-run: would READ ${label}` : `READ ok: mock fetch ${action.target}`,
    };
  }

  if (opts.dryRun) {
    return {
      ok: true as const,
      dryRun: true,
      detail: `dry-run: would MUTATE ${label}  body=${JSON.stringify(action.body ?? {})}`,
    };
  }

  const check = verifyConfirm(action, opts.confirm, opts.now ?? Date.now());
  if (!check.ok) return check;
  return { ok: true as const, detail: `MUTATE ok: ${label}` };
}

Reads skip the token path.

Dry-run never executes.

Side effects pay the confirm tax.

Keep dry-run in the same code path as execute. If dry-run uses a different classifier, you will ship a lie.

Step 5: Script the Agent Run

const getPage = {
  tool: "web_fetch",
  method: "GET",
  target: "https://example.test/help",
};

const tipForm = {
  tool: "form_submit",
  method: "POST",
  target: "https://example.test/tips",
  body: {
    note: "example tip text for a demo form",
    name: "",
    contact: "",
  },
};

const webhook = {
  tool: "webhook_post",
  method: "POST",
  target: "https://hooks.example.test/agent",
  body: { event: "task_done", id: "demo-1" },
};

// 1. read passes
run(getPage);
// 2. mutate without confirm blocks
run(tipForm);
// 3. dry-run shows the body without executing
run(tipForm, { dryRun: true });
// 4. mint, then succeed once
const token = mintConfirm(tipForm);
run(tipForm, { confirm: token });
// 5. replay fails
run(tipForm, { confirm: token });
// 6. tip token does not unlock webhook
run(webhook, { confirm: mintConfirm(tipForm) });
// 7. matching webhook confirm succeeds
run(webhook, { confirm: mintConfirm(webhook) });
// 8. expired token fails
const signup = {
  tool: "form_submit",
  method: "POST",
  target: "https://example.test/signup",
  body: { email: "[email protected]" },
};
const old = mintConfirm(signup, Date.now() - TTL_MS - 1000);
run(signup, { confirm: old, now: Date.now() });

Those URLs and body fields are example inputs. Nothing is sent on the network.

Run it:

npx tsx confirm.ts

Real output from this machine:

tiny-confirm-gate demo
No API key. No network. Example payloads only.

[OK] 1. GET help page
         READ ok: fetched https://example.test/help (200, mock html)
[BLOCKED] 2. submit tip form (no confirm)
         missing confirm token  needConfirm hash=74739fa798c4…
[OK] 3. dry-run tip form
         dry-run: would MUTATE form_submit POST https://example.test/tips  body={"note":"example tip text for a demo form","name":"","contact":""}

[minted] confirm token for tip form (truncated)

[OK] 4. submit tip form (with confirm)
         MUTATE ok: submitted form → https://example.test/tips
[BLOCKED] 5. replay same token
         confirm token already used
[BLOCKED] 6. webhook with tip-form token
         confirm token does not match action hash
[OK] 7. webhook with matching confirm
         MUTATE ok: posted webhook → https://hooks.example.test/agent
[BLOCKED] 8. submit with expired token
         confirm token expired

== audit ==
allow web_fetch GET https://example.test/help  (read)
block form_submit POST https://example.test/tips  (missing confirm token)
dry   form_submit POST https://example.test/tips  hash=74739fa798c4…
mint  form_submit  hash=74739fa798c4…  ttl=60000ms
exec  form_submit POST https://example.test/tips  (confirm accepted (one-time))
block form_submit POST https://example.test/tips  (confirm token already used)
mint  form_submit  hash=74739fa798c4…  ttl=60000ms
block webhook_post POST https://hooks.example.test/agent  (confirm token does not match action hash)
mint  webhook_post  hash=e75b37df989c…  ttl=60000ms
exec  webhook_post POST https://hooks.example.test/agent  (confirm accepted (one-time))
mint  form_submit  hash=4dd90af7f071…  ttl=60000ms
block form_submit POST https://example.test/signup  (confirm token expired)

Reads pass. Mutations earn the token. Replay dies. Wrong hash dies. Expiry dies.

That is the whole product surface of a confirm gate.

Where It Breaks Down

Classification can lie. A GET that triggers a server-side delete is still a side effect in the real world. Pair method rules with tool allowlists and known-safe hosts.
The mint step is the hard part. A demo calls mintConfirm. Production needs a human UI, a dual-control policy, or a second system that never shares the agent's tool belt.
Hashing JSON is fragile. Key order, whitespace, and floating point will bite you. Canonicalize on purpose. Version the canonical form.
TTL and clocks. Clock skew between mint and gate will look like expiry. Prefer short TTLs and a shared clock source.
One-time use needs durable storage. An in-memory Map dies with the process. Use a shared store with atomic consume, or you will double-fire after a restart.
Dry-run is not a sandbox. It shows intent. It does not prove the downstream API is safe. Keep egress and confirm separate.
Ambiguous task instructions will still pressure the model. Anthropic notes that unclear scope and impossible tasks push workarounds. Your gate does not fix the prompt. It stops the mutate when the confirm is missing.

The Bigger Idea

Agent proposes action
  ↓
Classify (read | side_effect)
  ↓
Read → execute
  ↓
Side effect → hash payload
  ↓
Mint confirm (human or policy)
  ↓
Verify (signature, hash, TTL, one-time)
  ↓
Execute, or block and audit

The classifier provides the lane.

The hash provides the exactness.

The mint provides the judgment.

The verify step provides the stop.

The audit log provides the evidence.

This part is prediction, not history.

I think agents on the live web need the same habit good operators already have: look freely, change only with a confirm of what you are about to change.

Egress tells you the neighborhood.

Confirm tells you the deed.

When the model is wrong, you want a stop that does not depend on the model's mood.

When the model is right, you want a receipt for what was allowed.

Your agent hit the real web. Make every side effect earn the click.

Software should explain itself.

I'm building Helix around this idea: every change should come with the why. What changed, what got better, what got worse, and the evidence behind it.

What critical engineering knowledge is your team losing right now?

See what Helix reveals →

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.

← News 🤖 Ai 🏠 Home