Dev.to Security 🔐 Cybersecurity 👁 0 📖 5 min read

Nothing is stale, and the agent touched the wrong environment

A stop condition that asks "is this approval stale?" catches nothing in the case that matters most. I have been arguing this in a thread about agent stop conditions, and the argument only becomes useful if someone can ru

A stop condition that asks "is this approval stale?" catches nothing in the case that matters most. I have been arguing this in a thread about agent stop conditions, and the argument only becomes useful if someone can run it. So here is the fixture, self-contained and fictional.

The case, in one paragraph

An approval names account A, pipeline P, and the production resource path. Scope version v4. The agent's run starts with scope v4 and stays inside the entitlement it was given. The runner resolves its alias and lands on staging. Nothing is stale. Every record is current. The two records are internally consistent with each other. The approval says one target, the execution hit another, and a gate that validates freshness, version, signature or label sees a clean pass on both sides.

That is the failure worth designing for: not an expired approval, a wrong one, executed exactly as written.

Why "same version" is the whole point

Most gate designs pick an invalidation axis they can compute cheaply. Scope version is the popular one, because it is a field, it is monotonic, and a bump is easy to detect. The problem is that the axis only fires when something changed, and the interesting failures do not require a change. Two environments under one entitlement, one alias resolution, one pipeline definition reused across a promotion path, and you have a mismatch with no version delta to point at.

If your hold has to answer "which version invalidated this approval", it will never fire here. The fixture will pass, and you will read that pass as coverage.

What the hold must assert instead

Identity, in the vocabulary that exists outside the repository:

  • the object named in the approval (account, resource path, environment, deployment target), written as the ticketing system writes it
  • the object named in the execution, written as the runner writes it, before any human-readable normalisation
  • the comparison rule that maps one to the other, with a version, and a person who owns changing it

Expected output is not "held". It is "held, naming this pair as mismatched, under this rule at this version, owned by this person". A hold that cannot name the pair has not decided anything; it has refused.

The fixture

Fully fictional. One account, one pipeline, two environments, no real system names, nothing that came from a real customer.

{
  "fixture": "same-version-target-mismatch",
  "account": "acct-A",
  "pipeline": "P",
  "scope_version": "v4",
  "entitlement": "unchanged for the whole run",
  "approval_record": {
    "writer": "ticket-system identity",
    "approved_target": {"account": "acct-A", "path": "/data/exports/prod", "environment": "production"},
    "scope_version": "v4"
  },
  "runtime_record": {
    "writer": "runner identity",
    "resolved_target": {"account": "acct-A", "path": "/data/exports/stage", "environment": "staging"},
    "scope_version": "v4",
    "alias_resolution": "exports -> stage"
  },
  "expected": {
    "decision": "deny",
    "must_name": ["approved_target", "resolved_target", "rule_id", "rule_version", "rule_owner"],
    "must_not": "deny solely because scope_version is stale"
  }
}

Positive control: the same run with the alias resolving to the production path. Expected allow, and the approval record and the runtime record still must be separable and separately attested. If the control case fails, the fixture is broken, not the system.

Negative control: no approval at all. Expected deny for a different reason, and the comparison is between the reason codes the gate records internally, not the message a user sees. A public denial can stay deliberately generic so it does not leak target details, and string-comparing the outward messages would fail a correct system. Distinct internal codes plus a generic outward message is the pass. What fails is both cases producing the same internal code, because then "nothing approved" and "approved the wrong target" are one event to the system.

Three acceptance checks, kept separate

  1. Dispatch. Whether execution started is observed from the executor, not from the agent's own log. A hold that logs is a claim. And if the only account of the resolved target comes from the runner's own log, the result is unknown, not deny: an unsupplied alias resolution is not a detection.
  2. Recomputation. Someone who was not in the room rebuilds the decision offline from the exported records: attempted target, rule, rule owner. If that is not possible, the export is a narrative.
  3. Provenance. The separation between the two records is attested by each record's own system of record, not by a field that says who wrote it.

Check 3 is the one I got wrong first, and a commenter in that thread corrected me properly. I had said the fixture should require "different writers". That is necessary and not sufficient: two records authored by the same team can still agree with each other, and a consistent pair written by the runner's team proves consistency and nothing else. What has to be verifiable is that the approval record's authorship comes from the ticketing system's own audit trail and the runtime record's from the runner's, each independently checkable. Labels are not provenance.

Four ways a fixture fools itself

  • The expected hold names a reason the case cannot produce ("stale scope") so the case passes as a false negative and reads as a win.
  • The mismatch is planted in a comment or a variable name rather than in the resolved target, so the system under test never had a chance to see it.
  • The two records come from one writer, and the "independent" recomputation is a re-read of the same source.
  • The negative control is missing, so a system that denies everything scores perfectly.

If you run this

I want the boring result, not agreement: what your gate returned, whether dispatch started, and what the receiver or executor saw. This fixture has no executed run behind it in my hands. It is a shape, published so it can stop being a comment. If it turns out the mismatch is undetectable without a change to how the runner resolves aliases, that is the finding, and it is worth more than a pass.

The discussion that produced this shape is public on DEV. I am keeping the fixture generic on purpose: no bank, no vendor, no customer.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.