Dev.to AI 🤖 Ai 👁 0 📖 17 min read

Make your repo enforce what you tell your coding agent: guard tests, ratchets and rule IDs you can copy

TL;DR Your coding agent treats CLAUDE.md as context, not law. Anthropic's own docs say so. Every rule you only write down will drift, at agent speed. Turn each convention into a test whose failure message is the conven

TL;DR

  • Your coding agent treats CLAUDE.md as context, not law. Anthropic's own docs say so. Every rule you only write down will drift, at agent speed.
  • Turn each convention into a test whose failure message is the convention. Then the agent's strongest habit (make the red build green) enforces your rule for you.
  • Copy five shapes: rule IDs with an Enforced by line, the census (derive the population, check both directions), the monopoly (only one module may do the dangerous thing), the two-sided ratchet, and debt with a deadline. Add a review ledger for prose.
  • Keep CLAUDE.md a map: where things are, and the checklist whose every step names the test that fails if it's skipped.
  • When you correct an agent for the same thing twice, stop writing sentences. Write the test.

I build Gitmoap, a code forge I run on my own: git hosting, review, a work tracker, pipelines as code, package registries, an MCP server. In nine months it reached 865 HTTP operations, 442 spec files and 243 migrations, all of it built with Claude Code, from the first commit. This post is the setup that made that possible, as templates you can paste into your own repo. The examples are from Gitmoap's Rust codebase, simplified; the Python ones are sketches written for this post, not Gitmoap code.

The problem in one table

Every rule in my repository has an ID and a line saying how it's enforced. That let me check, for this post, what happened to rules depending on how they're enforced:

Rule Enforced by State today
A released migration is never edited a checksum file and a test none edited since the test arrived; 243 migrations
Every endpoint group has a docs page a test over the router's registry 0 missing, exemption list empty
No delivery-phase labels in comments "code review; a grep audit" 29 such comments, 2 of them written after the rule
No unwrap() in production code a lint "once adopted" not adopted

Same author, same care, same agents. The rules that can fail the build hold. The others are wishes. Anthropic's documentation for Claude Code says it directly: CLAUDE.md and auto memory are loaded into every session, and "Claude treats them as context, not enforced configuration."

So the work is converting wishes into tests. Here's how.

1. Give every rule an ID and an Enforced by line

Put your rules in a few markdown files by theme, one heading per rule, with a stable ID:

### SEC-AUTH-FAIL-CLOSED — Authentication is fail-closed; the dev fallback is a positive opt-in.
- **Why:** a header fallback gated on "the pool id is unset" fails open: one missing env var
  turns the whole API into header-trust.
- **How:** the fallback needs a positive dev flag. A surface that verifies tokens refuses to
  start without a validator unless that flag is set.
- **Enforced by:** a startup assertion; an integration test that boots the production
  profile without a validator and asserts it refuses to start.

Three conventions make this pay:

  • IDs are forever. A retired rule leaves a tombstone heading (### RUST-CFG-OS *(retired)* — …); a renamed rule keeps its ID. So a comment, a test name or a commit message can cite SEC-AUTH-FAIL-CLOSED and still mean the same thing in a year.
  • Cite IDs in code. /// Fail-closed (SEC-AUTH-FAIL-CLOSED): … above a function gives the agent the reason, not just the code. In Gitmoap 171 distinct rule IDs are cited in the Rust source.
  • The Enforced by line is the audit. Grep it. Mine has 265 such lines; a rough regex finds about 114 naming a mechanical check and about 125 naming only review, and 24 say, in full, "Enforced by: PR review." in a repository that commits straight to main. That's your worklist.

Then make citations checkable. Gitmoap has a full markdown grammar for this (its spec index runs on customers' repositories too), but the first 80% is a short script:

#!/usr/bin/env python3
"""Fail when anything cites a rule ID that no spec declares (sketch)."""
import pathlib, re, sys

FAMILY = r"(?:ARCH|RUST|SEC|TEST|OBS|PERF|DOCS)"
DECLARE = re.compile(rf"^#{{2,4}} ({FAMILY}-[A-Z0-9-]+)\b.*? — ", re.M)
CITE = re.compile(rf"\b({FAMILY}-[A-Z0-9]+(?:-[A-Z0-9]+)*)\b")

declared: dict[str, pathlib.Path] = {}
for path in pathlib.Path("specs").rglob("*.md"):
    for m in DECLARE.finditer(path.read_text()):
        rule = m.group(1)
        if rule in declared:
            sys.exit(f"{rule} is declared twice: {declared[rule]} and {path}")
        declared[rule] = path

dangling = []
for root in ("specs", "src", "tests"):
    for path in pathlib.Path(root).rglob("*"):
        if path.suffix not in {".md", ".rs", ".py", ".ts"}:
            continue
        for n, line in enumerate(path.read_text().splitlines(), 1):
            dangling += [f"{path}:{n}: {r}" for r in CITE.findall(line) if r not in declared]

if dangling:
    sys.exit("cited but never declared:\n" + "\n".join(dangling)
             + "\n\nDeclare the rule under a heading in specs/, or fix the citation.")
print(f"{len(declared)} rules declared, every citation resolves")

Run it in CI and in the test suite. An agent that invents SEC-TOKEN-ROTATION in a comment now has to either declare it (and write its Why, How and Enforced by) or stop citing it.

One lesson I paid for: never key a check on a section number. My first spec guard found the "Simulation Coverage" section by matching its heading as a literal string, number included. Thirty-six of a hundred documents numbered that section differently, so the guard never opened them, none declared what it was supposed to, and the build stayed green. Identify sections by name.

2. The census: derive the population, check both directions

The most valuable guard shape. It reads the real list of things from the code (never a hand-kept list), and asserts three things: everything is covered, everything claimed still exists, and every exemption is still needed.

Here's Gitmoap's documentation census, simplified. Every endpoint group must be claimed by a docs page, through the tags: in the page's frontmatter:

/// Endpoint groups no page claims: (group, why). Shrink it; never grow it.
const UNDOCUMENTED: &[(&str, &str)] = &[];

fn served_groups() -> BTreeSet<&'static str> {
    // The same registry the router is built from, so a new endpoint can't be missed.
    gitmoap::web::endpoint_registry::all().map(|e| e.tag).collect()
}

#[test]
fn every_served_group_has_a_page() {
    let claimed = groups_claimed_by_pages();
    let exempt: BTreeSet<_> = UNDOCUMENTED.iter().map(|(g, _)| *g).collect();
    let missing: Vec<_> = served_groups()
        .into_iter()
        .filter(|g| !claimed.contains_key(g) && !exempt.contains(g))
        .collect();
    assert!(missing.is_empty(),
        "endpoint groups with no documentation page: {missing:?}\n\
         Write a page and list the group in its frontmatter `tags:`. If it genuinely cannot \
         be written yet, add it to UNDOCUMENTED with a reason a reader can disagree with.");
}

#[test]
fn every_claimed_group_is_still_served() {
    let served = served_groups();
    for (group, page) in groups_claimed_by_pages() {
        assert!(served.contains(group.as_str()),
            "{page} claims `{group}`, which the platform no longer serves. \
             A page documenting something removed is rot that reads as coverage.");
    }
}

#[test]
fn no_exemption_is_stale() {
    let claimed = groups_claimed_by_pages();
    for (group, reason) in UNDOCUMENTED {
        assert!(!reason.trim().is_empty(), "`{group}` is exempt with no reason");
        assert!(!claimed.contains_key(*group),
            "`{group}` is documented now; remove it from UNDOCUMENTED");
    }
}

The same three tests work in any stack. With FastAPI the population is one comprehension:

from fastapi.routing import APIRoute
from myapp.main import app

def served_tags() -> set[str]:
    return {t for r in app.routes if isinstance(r, APIRoute) for t in (r.tags or [])}

Why a list of exemptions rather than a count? The comment above Gitmoap's list says it better than I can: "A number says fifty-five groups are undocumented and nothing else; the list says which, and requires a sentence per entry that a reader can disagree with."

Gitmoap runs this shape everywhere: every operation placed in the feature register or excluded with a typed reason; every new database column that names a person classified for account deletion; every new DEFAULT now() column classified for the simulator; every queue consumer routed in the simulator's event bus. Each one is a question somebody would otherwise forget to ask, asked by the build.

3. The monopoly: one module does the dangerous thing

Some operations must only ever happen in one place. In Gitmoap a branch moves for a platform-written commit only with a PushClearance, and only the push gate may create one, after the branch rules, push rules and storage quota have all said yes. Visibility can't express that across crates, so a test does:

const THE_GATE: &str = "services/push_gate.rs";

#[test]
fn only_the_push_gate_mints_a_clearance() {
    let mut offenders = Vec::new();
    for file in workspace::rust_files(&workspace::platform_src()) {
        if workspace::module_rel(&file) == THE_GATE { continue; }
        let src = std::fs::read_to_string(&file).unwrap_or_default();
        for (at, _) in src.match_indices("PushClearance::mint(") {
            let line = src[..at].matches('\n').count() + 1;
            offenders.push(format!("  {}:{line}", workspace::rel(&file)));
        }
    }
    assert!(offenders.is_empty(),
        "a push clearance is minted outside the push gate:\n{}\n\n\
         Ask the gate instead — `push_gate::clear` — so the branch rules, the push rules \
         and the storage quota have each said yes before a branch moves.",
        offenders.join("\n"));
}

/// Otherwise the test above guards nothing.
#[test]
fn the_gate_is_where_a_clearance_is_minted() {
    let src = std::fs::read_to_string(workspace::module_file(THE_GATE)).unwrap();
    assert!(src.contains("PushClearance::mint("), "has the gate moved? Update THE_GATE.");
}

The second test is the part people skip. A text-scanning guard goes silently green when the thing it scans for is renamed or moved. Every guard should also assert that its subject still exists.

Two more things this shape needs to be fair: put the reason in the failure message (the agent will read it and do what it says), and say what to do instead (Ask the gate instead — push_gate::clear).

4. The two-sided ratchet

For debt you can't clear today, a budget that only falls. Most ratchets are one-sided: fail when the count goes over. That's half of it. When the count drops and nobody lowers the budget, the ceiling stops measuring anything. Gitmoap's version fails both ways, with a message for each:

pub const RATCHET_SLACK: usize = 5;

pub enum Ratchet {
    Ok { actual: usize, budget: usize },
    Over { actual: usize, budget: usize },
    Slack { actual: usize, budget: usize },
}

impl Ratchet {
    pub fn judge(actual: usize, budget: usize) -> Ratchet {
        if actual > budget {
            Ratchet::Over { actual, budget }
        } else if budget.saturating_sub(actual) > RATCHET_SLACK {
            Ratchet::Slack { actual, budget }
        } else {
            Ratchet::Ok { actual, budget }
        }
    }

    pub fn message(&self, name: &str) -> Option<String> {
        match self {
            Ratchet::Ok { .. } => None,
            Ratchet::Over { actual, budget } => Some(format!(
                "{name} is {actual}, over its budget of {budget}. This change added debt; fix it, \
                 or raise the budget deliberately with the full list in the change")),
            Ratchet::Slack { actual, budget } => Some(format!(
                "{name} is {actual} against a budget of {budget}. Lower the budget to {actual} so \
                 the ratchet keeps its grip — a budget that drifts above the real number stops \
                 being one")),
        }
    }
}

Keep a movement log next to the constant: every time the budget moves, a line in a comment saying why, in the same change. Gitmoap's budget for "operations served with no spec heading of their own" went from 181 to 216 over three months, and every step is explained above the constant. A ratchet that climbs is allowed. A ratchet that climbs silently isn't.

And when a check reaches zero, make it a plain assertion in the same change. A check installed at zero costs one constant; the same check installed a week earlier costs a budget, a movement log, and everyone's attention every time the number moves.

5. Debt with a deadline

A staged migration (expand, migrate, contract) gets stuck at the last step for a structural reason: expand and migrate are pulled by a feature, and contract is pulled by nothing. A checklist doesn't decay. An allowlist that may only shrink doesn't either: one of mine went from six entries to two, then sat at two indefinitely.

So the debt carries a date, in the file that created it:

-- gitmoap:expand drops=environments.secret_path_prefix by=2026-11-05 one N-2 release window past E4 …

and the later migration that pays it says so:

-- gitmoap:contract discharges=environments.secret_path_prefix one N-2 window has passed, no binary in service reads or writes the column …
ALTER TABLE environments DROP COLUMN secret_path_prefix;

The test is small. A sketch:

import datetime, pathlib, re

EXPAND = re.compile(r"--\s*gitmoap:expand drops=(\S+) by=(\d{4}-\d{2}-\d{2})")
CONTRACT = re.compile(r"--\s*gitmoap:contract discharges=(\S+)")

def test_no_contraction_is_overdue(today=datetime.date.today()):
    sql = "\n".join(p.read_text() for p in sorted(pathlib.Path("migrations").glob("*.sql")))
    discharged = set(CONTRACT.findall(sql))
    overdue = [f"{target} (due {due})" for target, due in EXPAND.findall(sql)
               if target not in discharged and datetime.date.fromisoformat(due) < today]
    assert not overdue, ("overdue contractions: " + ", ".join(overdue) +
        "\nWrite the migration that drops them, or move `by=` in a commit that says why.")

In Gitmoap that column's by= was 5 November. The build went red on the 6th; migration 0194 dropped the column on the 7th. Moving the date is allowed, but it happens in a commit, so "not yet" stays distinguishable from "never".

The general rule from the lint's header is worth pinning above your desk: "A rule you cannot opt out of gets deleted; one that makes you write down why gets obeyed." Every hatch (-- gitmoap:contract, -- gitmoap:lock, an exemption list entry) costs a written reason.

6. A review ledger for prose

Code drifts and the build notices. A spec can be rewritten in the same commit as the behaviour and still claim it was "last reviewed" eight months ago. Gitmoap's fix is specs/REVIEWED: one line per document, the SHA-256 of its body (frontmatter excluded, so stamping the review date can't invalidate the digest), regenerated by a bless command and checked in CI. A minimal version:

#!/usr/bin/env python3
"""specs/REVIEWED: bless with `review.py bless`, check with `review.py check` (sketch)."""
import hashlib, pathlib, sys

LEDGER = pathlib.Path("specs/REVIEWED")

def body(text: str) -> str:
    if text.startswith("---\n"):
        text = text.split("\n---\n", 1)[1]          # drop the frontmatter
    return text.replace("\r\n", "\n").strip()

def digests() -> dict[str, str]:
    return {str(p): hashlib.sha256(body(p.read_text()).encode()).hexdigest()
            for p in sorted(pathlib.Path("specs").rglob("*.md"))}

if sys.argv[1:] == ["bless"]:
    LEDGER.write_text("".join(f"{d}  {p}\n" for p, d in digests().items()))
else:
    recorded = dict(reversed(l.split("  ", 1)) for l in LEDGER.read_text().splitlines())
    moved = [p for p, d in digests().items() if recorded.get(p) != d]
    sys.exit(f"changed since last review, re-read and bless: {moved}" if moved else 0)

Be honest about what it proves. It doesn't prove anyone read the document; a command gets run reflexively. What it buys is that the acknowledgement lands in the same diff as the edit. That's weaker than "reviewed" and much stronger than a date someone types.

Gitmoap applies the same idea to user docs against the API: each folder of pages has a generated CONTRACTS file listing, per page, every fact of every operation the page covers (method, path, scope, each parameter, each response field and type, each documented error). When audit records gained an outbound field, the regenerated seal showed this in the folder of the page that owed an edit:

 list_admin_audit_log  res.200.application/json.records![].operation_id!  string
+list_admin_audit_log  res.200.application/json.records![].outbound  object
+list_admin_audit_log  res.200.application/json.records![].outbound.oneOf[1].group_id!  string <uuid>

The test fails until the seal is regenerated, so the field-level change sits in the diff next to the page. The agent session that made the change added a row to the page in the same commit. A projection you can read beats a hash: + outbound object in a review is a sentence, where a changed digest isn't.

7. CLAUDE.md as a map, not a manual

The temptation is to put every rule in CLAUDE.md. Mine reached 19,633 words by July. I cut it to 3,512 the same day and gave it the line it still opens with: "This file is deliberately thin: specs/ is the single source of truth." (It has grown back to nearly 17,000 words, which is one of my open problems; Anthropic suggests under 200 lines.)

What belongs in it is what an agent needs to find its way and what a guard can't say. A skeleton:

# CLAUDE.md

One paragraph: what this is, and that `specs/` is the source of truth. Follow spec pointers.

## Commands
`make help` lists everything. The ones you need: check, lint, the test tiers,
the regenerators (openapi, census, docs seal), the bless commands.
Never run raw build tools; never run the whole-tree formatter.

## Where things live
One line per top-level directory and per crate/package: what it owns, its spec.

## Rules
Rules live in specs/engineering/*.md with IDs. Cite the ID in code and review.
Don't restate rules here.

## Adding a new endpoint (or consumer, or table)
Numbered steps. Every step names the test that fails if it's skipped:
1. Model …                      — `openapi::tests` refuses an undescribed schema
2. Migration … then `make migrations-bless`   — `migrations_lint` refuses an unblessed file
3. Declare the endpoint …       — the compiler refuses a missing facet (E0046)
4. Place it in features/ …      — `features_coverage` refuses an unplaced operation
5. Docs page …                  — `docs_coverage` refuses an undocumented group

## When you finish
Run the gates once, at the end, cheapest first. Fix every red gate you meet,
including ones you didn't cause. Regenerate, bless, bump package versions. Don't commit.

The checklist is where most of the value is, and also where most of my failures came from. Gitmoap's queue consumers need their name in a deploy list. The step was in a skill but not in CLAUDE.md's checklist, and sessions following CLAUDE.md skipped it three times; those consumers ran a placeholder that failed every invocation until a check caught it. An agent follows your checklist exactly. If the checklist is wrong, that's what it follows.

8. Promote corrections: from memory to guard

Claude Code keeps per-project memory. Mine has 241 notes: 52 are feedback ("don't do that again"), 104 are gotchas. Reading them back, the pattern is unmistakable. Here are some, by what eventually stopped them:

Correction How often What it became
Don't run two cargo builds at once (they fight over the lock) raised six times; once a wait loop stalled nine hours a lockf lock every build command goes through when agents run in parallel
Fix every red gate you meet, even ones you didn't cause after an agent called a failing guard "pre-existing" still a told rule, plus "run every root guard" (nine seconds)
Never run the whole-tree formatter once was enough: 1,079 files rewritten for a 34-file change told. A PreToolUse hook would make it mechanical, and I should write one
Every create needs a quota twice half-mechanical: the endpoint macro won't compile a write without a quota = answer, but it can't check the answer is honest
Unfinished migrations must decay once the by= date in section 5
Don't carry request context in a task-local once told; I chose 441 explicit edits over one task-local instead

The rule I follow now: the second time I correct the same thing, it becomes a test, a lock or a narrower command, never another sentence.

A note on hooks. Anthropic's docs recommend a PreToolUse hook when you need to block an action regardless of what the agent decides. I use none: everything above lives in the test suite, which binds me, CI and every agent alike. Hooks are the right tool for actions (don't run this command); tests are the right tool for states (the repository must never look like this). The formatter row above is an action, which is why it's still only told.

9. What this costs

  • Build time. The guards are cheap (Gitmoap's root suite runs in about nine seconds once built), but the guard binary links the platform. A cold full unit run was 47 minutes before I split the main crate and is about 16 now. Agents love running gates after every step; one of my notes exists because I told one we were "wasting tens of minutes". Run them once, at the end, cheapest first.
  • Guards only guard if they run. One commit landed with two red guards because its session ran three hand-picked suites. Make "run all of them" the cheap default.
  • A ledger proves less than it looks like. My tenets file passes its review check every day while recommending a database design I abandoned in July.
  • Writing the test is the real work. It's slower than writing the sentence. It's faster than finding the drift six months later.

Try it on your repo

You can do all of this in any repository with a test runner. If you want the spec half without writing the scripts, Gitmoap does it on every push: put markdown under specs/, docs/, legal/ or blog/, and it indexes every rule ID you declare and everyone who cites it, every link between documents that goes nowhere and every unchecked box, and posts a gitmoap/specs check that goes red on a merge request linking to a document that isn't there. The guide: https://docs.gitmoap.com/guides/specs. A small TypeScript library set up this way, with rule IDs its tests cite by name: https://gitmoap.com/gitmoap-showcase/commitkit

Gitmoap is free, run by one person, stores code in AWS Ireland, and never uses your content to train generative models. Clone and push go through its own git helper (no plain HTTPS or SSH yet), and there's no importer yet. https://gitmoap.com

What's the rule your agent keeps breaking? I'd bet it's one that can't fail your build.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.