Why This Boring-Looking Diagram Actually Matters for AI

At first glance, this looks like another dry corporate diagram full of boxes, arrows, and numbers.

You might think: “Why should anyone care?”

But underneath those boxes is a simple idea: AI agents need boundaries around what they can see, what they can pass onward, and which sources they are allowed to trust.

Sentinel's Checkpoint V4 is one concrete experiment around that idea.

1. The big AI problem: giving an agent the keys to the house

When an AI agent can read files, write code, run commands, or use external tools, it is no longer just answering questions.

It is operating inside an environment.

That creates a strange security problem. The agent does not need to be intentionally malicious to cause trouble. It can misunderstand instructions, encounter hostile content, or treat data as instructions.

The important question therefore becomes:

What happens between the moment an agent encounters content and the moment that content reaches the model or gets acted upon?

That's where the three Sentinel layers come in.

1. What the three boxes actually mean

Layer 1: Detection

Sentinel scans content and labels suspicious material.

By Checkpoint V2, the benchmark detected 100 of 118 agent-payload cases, compared with 30/118 at baseline.

The remaining 18 misses were catalogued by mechanism rather than hidden behind a single score.

Importantly, detection is observational. The detector does not decide what the AI should do.

Layer 2: Containment

This is where V3 changed the pipeline.

Instead of treating every file through the same generic channel, the Executor scans content according to its natural channel.

That changed the benchmark's containment behavior:

51 → 26 verbatim payload egresses.

Thirty prose payloads that had previously remained suspect were reclassified as hostile and neutralised before egress.

The remaining 26 are not magically “safe.” They are known benchmark boundaries: 23 Python files where the payload sits inside a docstring, plus 3 extensionless external puzzle files.

That distinction matters.

Containment here does not mean “an impenetrable sandbox.” It means that the Executor has a defined point where hostile content can be stopped before it is passed onward.

Layer 3: Provenance

V4 added something different: an explicit trusted context source and fail-closed behavior.

The benchmark exposed a simple but important problem: context shipped through the working directory could otherwise become part of what the system reads.

After MR-J:

70/70 compromised explicit context sources were rejected.
0 prompts were built from them.

At the same time, clean trusted-context cases were still consulted 70/70.

So the principle is simple:

If the explicitly trusted source itself becomes untrusted, don't guess. Stop.

That's provenance.

1. Why the numbers matter

The interesting part isn't one giant “security score.”

The layers are measured separately.

Detection changed through MR-A…H.

Containment changed through MR-I.

Provenance changed through MR-J.

That makes it possible to see which mechanism changed which result instead of throwing everything into one mysterious AI-security number.

And several things remained constant across the benchmark:

• 0 network calls
• 0 commits
• 0 secrets in the prompt
• 0 payload persistence
• 0/780 memory-carry rows
• gitleaks: 0/118 agent-payload cases
• semgrep: 0/118 agent-payload cases
• "decision_changed": not measured

That last line is important.

This benchmark measures the pipeline's behavior. It does not claim to prove that Sentinel makes an AI agent “safe.”

1. The Lawn Shortcut Paradox

Now imagine an L-shaped sidewalk around a patch of grass.

A human immediately sees the shortcut.

The straight line across the grass is shorter.

But an AI agent does not experience the world through human intuition. It sees representations, constraints, tools, files, APIs, and objectives.

Give it a goal and enough autonomy, and the path it discovers may look completely alien to a human.

It might take an indirect route through a tool chain.

It might exploit an unexpected representation of data.

It might interpret something as an instruction that a human would recognise as “just a file.”

It might find a route that was never part of the designer's mental model.

That is the real “lawn shortcut” problem.

The agent is not necessarily taking the path a human would consider obvious. It is navigating the system according to the representations and affordances available to it.

And that is why security based only on human expectations can become fragile.

But this is a conceptual explanation, not a result measured by Checkpoint V4.

V4 measures something more concrete:

what happens when potentially hostile content enters the pipeline.

1. The important idea: don't make the AI perfect

The interesting approach isn't necessarily to build a smarter model that somehow understands every possible attack.

Instead, you can build explicit boundaries around the model.

Detect the content.

Classify it.

Respect the channel it came through.

Verify the provenance of trusted context.

And when an explicitly trusted source becomes untrusted:

stop rather than improvise.

That is a very different philosophy from asking an LLM to be the final security authority.

1. And what about the money?

This is where the diagram can easily be oversold.

Checkpoint V4 does not prove that expensive AI-security platforms are unnecessary.

It does show something more modest and more useful:

some security boundaries can be implemented deterministically, locally, and independently of another LLM.

That can reduce unnecessary model calls and keep sensitive material inside the system.

The goal isn't “replace everything with a Python script.”

The goal is:

don't use an expensive probabilistic model for a problem that can be handled by a small deterministic control.

Sometimes the smartest thing an AI system can do is not ask another AI.

Sometimes it should simply say:

“This source is not trusted. Stop here.”

And that is the quiet idea hiding inside all those boring boxes and arrows.

Technical:

Checkpoint v4 — three layers, three closed cycles

Sentinel commit: 6023539254e (master, after MR-J !82). All inputs and this document live outside the
repository. Benchmark results only — no safety claim for Sentinel, no failure claim for any other product.

1. Frozen numbers

checkpoint what changed agent payloads detected clean controls flagged verbatim egress of a payload to the LLM P3 instruction in system role

v1 baseline 30/118 0/36 19 (not measured)
v2 (MR-A…H, !73–!80) 8 narrow rules + canonical pass 100/118 0/72 51 60/60
v3 + MR-I (!81, channel) Executor scans on the file's natural channel 100/118 (label only, +30 suspect→hostile) 0/72 26 60/60
v4 + MR-J (!82, trusted context) explicit context source + fail-closed scan 100/118 (unchanged) 0/72 26 (unchanged) 0/60

Constant across all: gitleaks 0/118, semgrep 0/118 on agent payloads (scope, not failure); 0 network calls,
0 commits, 0 secrets in prompt, 0 payload persistence, 0/780 memory-carry rows; decision_changed = NOT MEASURED.

1. What each layer now says

Detection — closed at v2. 18 misses catalogued by mechanism (pointer 3, long context 4, narrative
authority 5, third-person demos 3, invoice identity 3); 15/18 are not lexical. No rule changed in MR-I or MR-J.

Containment — MR-I moved the label to the natural channel, so 30 prose payloads became hostile and are
neutralised before egress (51 → 26). The remaining 26 verbatim egresses are exactly: 23 Python files whose
payload sits in a docstring (suspect on code, no data-region for Python strings) and 3 extensionless external
puzzle files (channel code by conservative default). Both are known, deliberate boundaries of MR-I.

Provenance — MR-J closed the trust boundary the provenance benchmark exposed:

cwd-shipped SENTINEL_CONTEXT.md is never read (repo reads 70 → 0), instruction in system role 60/60 → 0/60;

a compromised explicit trusted source stops the run: 70/70 files SKIP context-untrusted, 0 prompts built,
scan recorded, nothing sanitised; clean twins under the same mode 70/70 consulted with the context verbatim;

pointer / URL / -in-data rows identical to prov3 (pointer never followed, URL never fetched).

The three layers are now separately measurable and were changed by separate MRs (detector rules A–H,
Executor channel I, Executor trust J). None of them authorises anything: InjectionGate labels stay observational,
the Executor stop is a refusal to consult, not a decision.

1. Open boundaries (not defects, candidates for a next cycle — each needs its own approval)

boundary evidence smallest honest next step risk

a Python docstring is not a data region 23/26 remaining egresses measure only: how many hazmat/benchmark .py files would change label if string literals were data regions (tree-sitter-python is already a dependency) label shift on .py controls; must be a dry run first
b extensionless / unknown files default to code 3/26 remaining egresses content sniffing would be a heuristic — probably document, do not implement FP on scripts without extension
c suspect files go verbatim by policy 26 egresses are all "not hostile" a policy decision (sanitise suspect? mark as data?) — not a detector question; needs a drift measurement with a non-stub model, i.e. decision_changed changes prompt content for every suspect file
d pointer / URL following Executor never follows; 3 pointer misses nothing to fix in Sentinel; the boundary is at a consumer that does follow none now
e 18 detection misses catalogue leave as catalogue —

Recommendation: (a) as a measurement-only dry run, then decide; (c) only together with a real
decision_changed experiment, which is a different kind of benchmark (non-deterministic model, cost, isolation).

End...
One last question for anyone building autonomous agents

You’re giving your agent access to your code, your files, your tools, and probably your cloud budget.

So here’s the uncomfortable question:

Have you actually tested what happens when the agent is lied to?

Not when the model follows your instructions perfectly.

When the input is hostile.
When the “trusted” context is compromised.
When the payload looks like data but behaves like an instruction.
When the agent finds a path you never expected.

Because the most expensive sentence in AI engineering might eventually be:

“I didn’t think the agent would do that.”

And by then, your API keys, your data, and your cloud bill may already have learned the lesson for you.

So before giving an agent more autonomy:

Test the boundaries. Not just the intelligence.