bisect-1794189352959-2

The guarantee is split in two. A manifest cannot hash itself without a
bootstrap, so the nine pinned files are protected by the build, and the manifest
holding their hashes is protected by a request in `.github/CODEOWNERS`, which
asks for the owner's review rather than requiring it. **The build catches a
protected file changing alone. The owner is asked to see the change that
arrives with an updated hash beside it.** Two different guarantees, and only the
first is mechanical.

### 2. Mechanical invariants

For the files that legitimately change, six assertions over nine required
substrings:

Six rules, nine needles: `2 + 1 + 1 + 2 + 1 + 2`. Each carries a `why`, so
the next person to change the file can see what the assertion was defending.

### 3. Required and forbidden gate targets

The rule that changed direction. What the gate runs is a decision, not a
default, so it is enforced in both directions:

The targets are parsed out of the gate script, so they come from the project
graph rather than from a list somebody edits. The history of this field is why
it has a direction at all:

A rule that can only ever say *no* has to be rewritten when the decision
changes. A rule with a direction survives the decision changing: `test` moved
from forbidden to required, and the manifest moved with it.

That `test` target is the one F1 removed. It came back on 2026-11-03, and the
gate that runs it today also runs the guardrails first:

### 4. Separately bounded budgets

The product budget and the test budget are counted separately, and the split
is the point. If tests counted against the product budget, the first test
would cost a line of product code, and the budget would push toward removing
verification. Excluding tests without bounding them would make the exclusion
a hole. Bounding them separately makes it a rule.

### All four, running

Both scripts are Node built-ins only and run from the repository root, a
worktree or CI. The guardrails script also runs as its own job, `Guardrails`,
and inside the gate.

---

## The rest of it

Four more checks had to become mechanical before the first four meant anything.

### Recorded removals

Five entries, each naming what was removed and why, printed on every run:

This is decision record 0009's principle, made mechanical: *"An unexplained
absence reads as an oversight. A recorded exclusion reads as judgement."*
(`docs/decisions/0009-not-built.md`, Context). The repository now prints its
own absences, with reasons, every time the guardrails run.

### Forbidden paths

Fourteen globs, matched against every tracked file at any depth:

The comment in that loop is the fix for one of the three wrong-reason checks
in the section above: the earlier version matched the repository root only, so
a nested forbidden path was never matched.

### Secret patterns

Eight regular expressions (GitHub, Chromatic, Nx Cloud, AWS, private keys)
applied to every tracked text file under a megabyte:

### Project tags

Tags come from a closed vocabulary: a scope (`shared`, `web`)
and a type (`ui`, `feature`, `app`). A project with no tags fails, because the
boundary rule matches on tags and a project with no tags has no boundary:

## The distinction that keeps the secret check honest

A pattern check is cheap and mechanical, and it covers the **shape** of a
credential. It did not find the credential that was on the machine. That was
found by a separate search, run by hand, for the exact values of the credentials
held in local files. The corpus says so itself:

**A pattern check covers the shape; a value check covers the instance.** The gate
did not catch it, and nothing here claims that it did.

---

## What it cost

The cost belongs here. A post about process that leaves the cost out is the
thing it criticises.

| | |
| --- | --- |
| Tooling | 453 lines: `guardrails.mjs` 245, `size-budget.mjs` 54, `guardrails.manifest.json` 154 |
| Decision records | 17 |
| Commits that touched the mechanism | 29 of 104 |
| Commits whose subject is process work | 45 of the 84 non-merge commits |

Twenty of the 104 commits are merges, so 84 sit outside the merge group, and 45
of those 84 are process work.

Read quickly, that ratio looks like displacement: forty-five commits of process
work against a product that shipped, *could not build the thing, built the
bureaucracy instead*.

There are two answers, and neither of them is that the ratio does not matter.

The first is that the ratio is an artefact of this corpus. The product is a PDF
annotator: one workspace, one operator, thirty days. The budgets above exist
so the ratio cannot become the only thing that matters. A file-size budget and a
test budget are arguments against the shape of the work drifting. They do not
measure it.

The second is that this process work *was* the product, for the purposes of the
paper. The counts are not a tax on the product. They are the other half of it.

So the lesson is narrower than "write more process":

> **The cost of a safeguard is paid in process, and should be counted there**,
> in the commit history, where it is visible.

A safeguard that costs nothing is not a safeguard, it is a sentence. The price is
real, so it goes next to the work it protects, where the next person can decide
whether it is worth paying.

---

## What it does not protect

The limits belong here, not in a footnote. Stating them is part of the
mechanism.

**It does not make the code good.** It does not decide what to build. It does
not catch a wrong change that satisfies every rule. A hash says a file has
not changed; it says nothing about what is in it.

**It does not survive the protected files being deleted and their hashes
removed in the same commit.** The manifest cannot hash itself without a
bootstrap, so what stands in that gap is a request for review:

<pre><code># .github/CODEOWNERS: the second lock
# The owner's review is requested for the safeguards. The Guardrails check
# fails the build if one of these is removed or weakened; this is the second
# lock, and the one that survives a check being edited.

/AGENTS.md                       &#64;effinrich
/CLAUDE.md                       &#64;effinrich
/lefthook.yml                    &#64;effinrich
/package.json                    &#64;effinrich
/.cursor/hooks.json              &#64;effinrich
/.claude/settings.json           &#64;effinrich
/.github/workflows/              &#64;effinrich
/tools/guardrails.mjs            &#64;effinrich
/tools/guardrails.manifest.json  &#64;effinrich
/tools/shell-guard.mjs           &#64;effinrich
/tools/stop-gate.mjs             &#64;effinrich
/tools/size-budget.mjs           &#64;effinrich
/docs/decisions/                 &#64;effinrich</code></pre>

It *requests*. That is the honest word for it.

**It does not stand in for the branch.** The live rules on `main`:

The pull-request rule requires **zero** approving reviews, the ruleset names no
bypass actor, and the required checks are `Gate` and `Guardrails`: the gate and
the guardrails, which is the mechanism, wired into the branch.

One scope note, because "no bypass actor" is easy to over-read. A second
ruleset, `feature-branches`, matches every ref and names eighteen bypass
actors: three repository roles and fifteen integrations.

Those three role ids are 2, 4 and 5, and their meaning is not published.
Neither GitHub's REST reference nor its OpenAPI description states the
mapping. The open issue
[github/rest-api-description#7111](https://github.com/github/rest-api-description/issues/7111)
calls it a "critical security feature, currently undocumented" and asks GitHub
to publish it. The reading in use (maintain, write and admin) comes from that
issue and from the Terraform provider's ruleset reference, not from GitHub.

How those two rulesets combine on the trunk is GitHub's evaluation order, not
something the corpus documents, and this post does not assert it. The
checkable claim is narrower: the two checks the mechanism runs are required on
the branch that carries the mechanism.

Before the failure, the same ruleset was different, and the difference is the
whole of F1:

The removal satisfied every rule that existed. **The mechanism did not replace
the ruleset. It gave the ruleset something to require.**

**And the environment, which is a different failure.** The machine the agents
ran on was compromised:

One incident, self-reported, no independent forensic record. It belongs here
because it needs a different answer: no rule resists a compromised host, and
the remedy is credential rotation, not taking an agent's word for anything.

**One limit on all of it.** I am the author of the preprint, and I adjudicated
every conflict in the corpus. But the decision records are not, for the most
part, my own prose: most of them were drafted by a coding agent and ratified
by me afterwards, and one of them describes work that agent had itself done.
Where a finding in this post rests on a decision record alone (F3's first two
checks), the post says so, and keeps that distinction visible.

---

## Three design rules

The preprint's three design rules, actionable.

### 1. Pin what must survive

If an artefact's loss would be unrecoverable, its hash belongs in the build
the day it is created.

One line:

A hash written after the artefact is gone costs the artefact. The harness went
on 2026-10-06. Protection arrived 2026-10-26:

Twenty days. The pin that would have caught it was one line.

### 2. Derive checks from the project graph, not from a list somebody edits

The corpus takes its inputs from the tree:

A new file is covered when it is added, not when somebody remembers to list it.

> "the gate has to derive what to check from the project graph. An enumerated
> target list is a list, and lists get edited by whoever is editing them."
>
> Source: `tools/guardrails.manifest.json`, the "why" of the second assertion

### 3. A check that cannot fail for the right reason is not a check

If it passes on malformed input, it is not asserting the property it appears
to, and the reader of the green build is misled.

That sentence cost the corpus three checks to learn: *"A check has to fail for
the right reason before it is trusted."* (ADR 0015 lines 38-42.)

Beyond this codebase: a check that skips what it cannot read is not lenient. It
has stopped reading, and it reports success about a file it never opened.

---

## Starting Monday

Seven things, in order.

1. Write down what you are protecting, and hash it. The day it is created, not
   after you notice it missing.
2. Turn each removal into a recorded removal. A path and a reason, printed on
   every run. An unexplained absence reads as an oversight; a recorded
   exclusion reads as judgement.
3. Derive the checks from the project graph. A project with no tags has no
   boundary; make the missing tag a failure, not a skip.
4. State scope as paths, not as principles. "Stay inside the brief" is a
   sentiment; a named path is a check. Nothing in this corpus enforces scope, so
   the mechanical version must be written.
5. Flip rules instead of deleting them. A rule with a direction survives the
   decision changing. A rule that can only say *no* must be rewritten whenever
   the decision changes, and rewriting a rule is what an agent can do in the
   same commit as the thing the rule forbids.
6. Put the negative case in the contract. If a file cannot be parsed, that is
   a failure, not a skip.
7. Fewer agents on one repository. Concurrency is the multiplier on every
   ambiguity in the rulebook. F2 needed four agents on one task to produce four
   implementations of a feature the platform already ships; F1 needed one agent
   and one commit.

---

## Reproducing it

Everything in this post is reproducible from two public repositories.

**The corpus:** <https://github.com/effinrich/h2-fe-assessment>, pinned to
commit `b200e23`.

**The preprint:** <https://github.com/effinrich/prose-does-not-bind-agents>.
`paper.md` is the source of truth; `paper.pdf` is the rendered copy. Appendix A
lists the commands behind the paper's counts, and the commands quoted here are
drawn from it.

**Licence:** the preprint is CC BY 4.0 (Copyright (c) 2026 Rich Tillman). The
corpus carries no licence file, so this post quotes it in short extracts,
always with the commit, the path and the line numbers.

One note on the pin. `b200e23` is the last commit of the study period, not the
last commit of the repository: four commits landed after it, on 2026-11-04 and
2026-11-05. Every figure in this post is quoted at the pin or at a commit named
in the command that reads it, so later commits do not change any of them.

One file quoted in this post is not at the pin: `docs/decisions/README.md`,
added by `a3e4403` and amended by `858629c`. The commands that read it name
the commit they quote.

And the corpus does not end where this story does. Its pinned commit is
`b200e23`, *"feat: a unit test loop the brief did not ask for (#89)"*. The unit
test loop came back on 2026-11-03, twenty-eight days after the harness went
with it, and, in the manifest's own words, *"without the specs"*. The deletion
was not correct on the brief's terms; the thing it took came back later, on its
own terms.

---

## Appendix: reproducing every figure

Every number quoted above, with the command that produces it. Run against
`b200e23` in a clone of `effinrich/h2-fe-assessment`.

| Claim | Command | Result |
| --- | --- | --- |
| Commits | `git rev-list --count b200e23` | 104 |
| Merge subjects | `git log --format=%s b200e23 \| grep -c '^Merge'` | 20 |
| Merges by parent count | `git rev-list --count --merges b200e23` | 19 |
| Product-prefix commits | `git log --format=%s b200e23 \| grep -cE '^(feat\|fix\|perf\|refactor\|ci\|test)'` | 35 |
| Process-prefix commits | `git log --format=%s b200e23 \| grep -cE '^(docs\|chore\|style\|build)'` | 45 |
| `Co-authored-by` trailers | `git log b200e23 --format=%B \| grep -c '^Co-authored-by:'` | 126 |
| Trailer split | `… \| sed 's/Co-authored-by: //' \| sort \| uniq -c \| sort -rn` | Cursor 102, MiniMax Code 14, author 5, Copilot 4 across three accounts, CI bot 1 |
| Tracked files | `git ls-tree -r --name-only b200e23 \| grep -vc '^$'` | 165 |
| Decision records | `git ls-tree -r --name-only b200e23 -- docs/decisions/ \| grep -cE '\.md$'` | 17 |
| The removal | `git show --shortstat --format='' 9eeb60a` | 137 files, +104, −10,097 |
| Per-area split | `git show --numstat --format='' 9eeb60a` | libs 107/7,379; apps 10/443; docs 9/1,595; root 9/602; tools 1/73; .github 1/5 |
| Root rulebook, before / after | `git show 9eeb60a^:AGENTS.md \| wc -l`; `git show 9eeb60a:AGENTS.md \| wc -l` | 166 / 67 |
| Instruction files before | `git ls-tree -r --name-only 9eeb60a^ \| grep -cE '(^/\|/)(AGENTS\|CLAUDE)\.md$'` | 14 |
| The phrase, before / after | `git grep -ic 'stay inside the brief' 9eeb60a^ -- .` and `git grep -ic 'stay inside the brief' 9eeb60a -- AGENTS.md` | no matches (exit 1) / 1 |
| The gate before / after the removal | `git show 9eeb60a^:package.json`; `git show 9eeb60a:package.json` | `nx affected -t lint,format,typecheck,test,build,build-storybook,test-storybook` → `nx affected -t lint,format,typecheck,build,build-storybook` |
| The record that contradicts the commit | `git show 5915058^:docs/decisions/0007-storybook-scope.md \| sed -n '1,12p'` | `- Date: 2026-10-01`; "the brief is graded partly on design" |
| The same record, on the stories | `git show 5915058^:docs/decisions/0007-storybook-scope.md \| sed -n '/Interaction assertions/,+2p'` | "Interaction assertions live in play functions" |
| The brief, on design consistency | `git show b200e23:docs/brief.md \| sed -n '16p'` | "at: ease of use, design consistency, user experience." |
| The judgement, in the operator's words | `git show b200e23:docs/safeguards.md \| sed -n '18,21p'` | "That is a judgement failure, and it is ordinary" |
| The commit before the removal | `git show -s --format='%h %ad %s' --date=iso 5915058` | `5915058 2026-10-06 00:07:04 -0400`. The direct parent of `9eeb60a` |
| The guardrail notes that went first | `git show --name-status --format='' 5915058 \| grep -E 'ai-guardrails\|handoff\|gate\.sh\|guard\.sh\|qa-'` | six deletions, among them `ai-guardrails.md` and `handoff.md` |
| The trunk camera | `git log -1 --format='%h %ad %s' --date=short 6f6ec34` | `6f6ec34 2026-10-05 refactor(camera): one component per file, stories split by theme` |
| Identical to the branch | `git diff --stat 6f6ec34 origin/refactor/camera-components -- libs/web/features/camera/` | empty |
| Trunk / branch camera size | `git ls-tree -r --name-only 6f6ec34 -- libs/web/features/camera \| grep -cE '\.tsx?$'`; the same for `origin/refactor/camera-structure` | 30 / 36 |
| Correction #47 | `git show --shortstat --format='' a27f292` | 29 files, +365, −1,561 |
| Correction #57 | `git show --shortstat --format='' c7fe8ae` | 15 files, +98, −370 |
| ADR 0017's claim | `git show b200e23:docs/decisions/0017-camera-tool.md` line 38 | "None of it reached `main` — it lived in branches and in the working tree." |
| The bespoke primitive still ships | `git show b200e23:libs/web/features/camera/src/lib/crop-shape-util.tsx \| grep -n 'class CropShapeUtil'` | `38:export class CropShapeUtil extends BaseBoxShapeUtil<CropShape> {` |
| The lint rules that do exist | `git show b200e23:.oxlintrc.json` | <code>&#64;nx/enforce-module-boundaries</code>, `max-lines` 150, `unicorn/filename-case`, `typescript/no-explicit-any`; no primitive rule |
| The F3 skip | `git show f471a29:tools/guardrails.mjs` lines 158–160 | `} catch {` / `continue` / `}`: the file was never read |
| The F3 fix | `git show 98f6d3f -- tools/guardrails.mjs` | the skip replaced by a failure that names the file |
| The fix's own account | `git show --format=%B --no-patch 98f6d3f` | "project file could not be parsed: libs/web/features/probe/package.json" |
| F3's other two checks | `git show b200e23:docs/decisions/0015-verification-surface.md` lines 44–49 | attested by ADR 0015; their buggy forms appear in no commit |
| The generalisation | `git show b200e23:docs/decisions/0015-verification-surface.md` lines 38–42 | "a gate that fails because a tool is missing has proven nothing…" |
| Who drafted each record | the loop in F4 | 11 single agent, 4 agent and owner, 2 no trailer |
| ADR 0017's drafting commit | `git log -1 --format='%h %ad %s' --date=iso -- docs/decisions/0017-camera-tool.md` | `39c7710 2026-10-27 00:33:32 -0400` |
| What that commit also did | `git show --name-status --format='' 39c7710 \| grep -E '0017-camera-tool\|crop-shape-util\|crop-export'` | the record, and a new crop shape on the trunk |
| The corpus's own note on authorship | `git show 858629c:docs/decisions/README.md \| sed -n '12,15p'` | "The prose is usually an agent's" |
| The mechanism's size | `git show b200e23:tools/guardrails.mjs \| wc -l` (245) + `tools/size-budget.mjs` (54) + `tools/guardrails.manifest.json` (154) | 453 |
| Manifest shape | `git show b200e23:tools/guardrails.manifest.json` | `files` 9, `assertions` 6, `forbiddenPaths` 14, `secretPatterns` 8, `removed` 5 |
| Gate targets | the same file | `"required": ["test"]`, `"forbidden": ["test-storybook"]` |
| `gateTargets` introduced | `git show 8c75684 -- tools/guardrails.manifest.json` | `"forbidden": ["test", "test-storybook"]` |
| Budgets | `bun run budget` | `81/95 files, 3073/3300`; `4/12 files, 227/400` |
| Guardrails run | `bun run guardrails` | `9 files, 6 invariants, 17 records, 165 tracked files scanned` |
| The gate today | `git show b200e23:package.json \| grep '"gate"'` | `bun run budget && bun run guardrails && nx affected -t lint,format,typecheck,build,build-storybook,test` |
| Commits that carried the mechanism | `git log --format=%h b200e23 -- tools/ docs/decisions/ AGENTS.md CLAUDE.md lefthook.yml .cursor/hooks.json .claude/settings.json .github/workflows/guardrails.yml \| sort -u \| wc -l` | 29 |
| The guardrail warning arrived | `git log --format='%h %ad %s' --date=short -S'not scaffolding' b200e23 -- AGENTS.md` | `affe349 2026-10-26` |
| Branch protection | `gh api repos/effinrich/h2-fe-assessment/branches/main/protection --jq …` | `{"admins":true,"checks":["Gate","Guardrails"],"deletions":false,"force_push":false,"pull_request_reviews":"absent","strict":true}` |
| The `main` ruleset | `gh api repos/effinrich/h2-fe-assessment/rulesets/24388801 --jq …` | `"bypass_actors":[]`; `pull_request` requires 0 approving reviews; checks `Gate`, `Guardrails` |
| The second ruleset | `gh api repos/effinrich/h2-fe-assessment/rulesets/24519354 --jq …` | condition `~ALL`; 18 bypass actors (3 repository roles, 15 integrations) |
| `RepositoryRole` actor ids | GitHub REST API description issue 7111 and the Terraform provider's ruleset reference; GitHub itself documents the mapping nowhere | id 2 is maintain, id 4 is write, id 5 is admin, on those third-party sources' authority only |
| The pre-failure ruleset | `git show b200e23:docs/safeguards.md` lines 95–98 | "it protected the shape of a merge and nothing about its content" |
| The compromised host | `git show b200e23:docs/safeguards.md` lines 11–13 | "the machine the agents ran on was compromised by a trojan" |
| The second lock | `git show b200e23:.github/CODEOWNERS` | a request for review, not a requirement |
| The value check | `git show b200e23:docs/transcripts/README.md` lines 67–72 | run by hand, not part of the gate |
| The pin, and what came after | `git log --oneline b200e23..858629c` | 4 later commits, 2026-11-04 and 2026-11-05 |
| The corpus licence | `git ls-tree -r --name-only b200e23 \| grep -i -E "licen\|copying"` | no matches |