Sign Off the Promises, Not the Paragraphs: A Claim-Strength Linter for Model-Drafted Docs

Uploads are atomic: the object is never visible in a partially written state.
Retries are idempotent when the client sends the same request id.
Run `mk upload ./dist` and wait for the final line to print `done`.

#!/usr/bin/env python3
"""Inventory promise-bearing sentences in model-drafted Markdown.

Usage:
  python tools/claims.py docs/*.md --signoff docs/claims.signoff.json
Exit code 1 when an unsigned claim at or above --min-severity exists.
"""
import argparse
import hashlib
import json
import re
import sys
from pathlib import Path

LEXICON = {
    "always": 3, "never": 3, "guarantee": 3, "guaranteed": 3, "zero-downtime": 3,
    "safe": 2, "safely": 2, "atomic": 2, "idempotent": 2, "supported": 2,
    "compatible": 2, "cannot": 2, "will not": 2,
    "instant": 1, "fast": 1, "secure": 1, "only": 1,
}

FENCE = re.compile(r"^\s*(```

|~~~)")
SPLIT = re.compile(r"(?<=[.!?])\s+(?=[A-Z`*])")


def sentences(path):
    in_fence = False
    for lineno, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        if FENCE.match(line):
            in_fence = not in_fence
            continue
        if in_fence or not line.strip():
            continue
        for piece in SPLIT.split(line.strip()):
            yield lineno, piece.strip()


def score(text):
    low = text.lower()
    hits = [t for t in LEXICON if re.search(rf"\b{re.escape(t)}\b", low)]
    if not hits:
        return 0, []
    return max(LEXICON[t] for t in hits), hits


def fingerprint(sentence):
    norm = re.sub(r"\s+", " ", sentence.lower()).strip()
    return hashlib.sha256(norm.encode("utf-8")).hexdigest()[:12]


def main(argv=None):
    ap = argparse.ArgumentParser()
    ap.add_argument("docs", nargs="+", type=Path)
    ap.add_argument("--signoff", type=Path, default=Path("docs/claims.signoff.json"))
    ap.add_argument("--min-severity", type=int, default=2)
    ap.add_argument("--root", type=Path, default=Path("."))
    args = ap.parse_args(argv)

    signed = json.loads(args.signoff.read_text(encoding="utf-8")) if args.signoff.exists() else {}
    findings, blocked = [], []

    for doc in args.docs:
        for lineno, sentence in sentences(doc):
            severity, hits = score(sentence)
            if severity == 0:
                continue
            fp = fingerprint(sentence)
            entry = signed.get(fp)
            status = "signed"
            if entry is None:
                status = "unsigned"
            elif not (args.root / entry["evidence"]).exists():
                status = "evidence-missing"
            findings.append({
                "file": str(doc), "line": lineno, "severity": severity,
                "terms": hits, "fingerprint": fp, "status": status,
                "sentence": sentence,
            })
            if severity >= args.min_severity and status != "signed":
                blocked.append(fp)

    print(json.dumps({"findings": findings, "blocked": blocked}, indent=2))
    return 1 if blocked else 0


if __name__ == "__main__":
    sys.exit(main())
```

Run it and the output is a flat list, not a verdict. Severity 3 means the sentence asserts an absolute, severity 2 means it asserts a property, and severity 1 is advisory noise you can ignore at first. The `blocked` array is what CI reads, and it stays empty until the docs actually contain unsigned promises.

``{% endraw %}{% raw %}`json
{
  "findings": [
    {
      "file": "docs/uploads.md",
      "line": 12,
      "severity": 3,
      "terms": ["never", "atomic"],
      "fingerprint": "<sha256-prefix>",
      "status": "unsigned",
      "sentence": "Uploads are atomic: the object is never visible in a partially written state."
    }
  ],
  "blocked": ["<sha256-prefix>"]
}
```

The example above is illustrative; the fingerprint value is abbreviated rather than computed on a real repository. What matters is that the same sentence produces the same fingerprint tomorrow, and that any wording change produces a new one.

This is where free model access on MonkeyCode fits the workflow rather than decorating it: the drafting pass in step 3 runs against the free models, and the nightly re-check in step 5 fits on the free server option, which is enough for a cron job that opens a pull request when evidence disappears. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

## 4. Record who is accountable for each promise

A sign-off is not a comment saying "LGTM"; it is a record that names an owner, points at evidence that exists in the repository, and carries the date a human last checked. Store it as JSON next to the docs so it diffs cleanly and needs no YAML parser in CI.

```json
{
  "<sha256-prefix>": {
    "claim": "Uploads are atomic: the object is never visible in a partially written state.",
    "owner": "storage-team",
    "evidence": "tests/test_upload_atomicity.py",
    "verified": "2026-11-07",
    "note": "Test asserts no partial object is readable during an aborted multipart write."
  }
}
```

Three properties matter here. The key is derived from the sentence, so rewording revokes the sign-off automatically rather than silently inheriting it. The evidence field is a path the script checks on every run, so deleting or moving the test breaks the build instead of the promise. The owner field is a team, not a person, because individuals rotate and docs outlive them.

## 5. Wire the check into CI and re-verify nightly

Add the check as a required job, then add a scheduled job that catches the slower failure: evidence that exists but no longer proves anything. The first job is fast; the second is the one that keeps the ledger honest after six months of refactors.

```yaml
- name: Promise sign-off check
  run: python tools/claims.py docs/*.md --signoff docs/claims.signoff.json
```

```bash
# nightly cron, free server tier
python tools/claims.py docs/*.md --signoff docs/claims.signoff.json --json \
  > /tmp/claims.json || true
jq -r '.findings[] | select(.status != "signed") | "\(.file):\(.line) \(.status)"' /tmp/claims.json
```

The nightly job prints only drifting claims, so a maintainer can triage ten lines instead of rereading the docs. One extension the script above deliberately omits is a `--max-age-days` flag comparing `verified` against today; add it once the team agrees how often a promise may go stale.

## What this does not catch

Be explicit about the gap, because the linter looks more authoritative than it is. It matches vocabulary, so a model that avoids the lexicon can still write a guarantee in softer words, and a quoted external guarantee will be flagged as if it were yours. Sentences split per line, so a doc that still hard-wraps paragraphs will produce mangled findings until the convention is adopted.

The larger gap is semantic: a signed claim with a test that asserts the wrong thing passes every check in this pipeline. The linter enforces that a human looked, not that the human was right, so pair it with the usual review of test intent. It also says nothing about whether the prose is worth reading, which remains an editorial judgment.

## Who should skip this

Teams whose documentation is fully generated from an OpenAPI schema or type signatures get nothing here, because there are no human-authored promises to sign. Solo projects with no test suite should not adopt the ledger either, since the evidence field would be decorative and the ceremony would outrun the value. Finally, skip it if nobody on the team will review a nightly drift report; an unread report is worse than no report, because it looks like coverage.

For everyone else the order is fixed: agree on draft rights, normalize sentences, inventory promises, sign them with live evidence, and let CI hold the line. The model keeps writing the parts that are cheap to be wrong about, and a human keeps signing the parts that are expensive. If you want to test the loop before paying for CI capacity, the drafting pass and the nightly re-check both run on the free tier described above.