A Rejection Ledger for Model-Drafted Docs: Codify Why Drafts Fail, Then Constrain the Next Run

Documentation pipelines usually break at review, not at generation, because the reasons a draft was rejected live in PR comments and disappear from context by the next run. The split that holds up is narrower: humans own a versioned constraint ledger, the model may draft only text that satisfies written constraints, and a small class of claims stays outside drafting entirely. Everything below is illustrative rather than measured; the code is a starting point you validate against your own repository before relying on it.

1. Why review comments cannot constrain the next draft

A comment like "do not promise a release date" is legible to one reviewer once and invisible to every subsequent generation. The same rule gets written again in the next PR, phrased slightly differently, and the model has no way to learn from either instance. Repeated failure of this kind is a storage problem before it is a prompting problem. Moving the rule into a file that both the prompt builder and a checker read converts folklore into a constraint.

2. The ownership split: three artifacts, three owners

Before writing any prompt, decide what each artifact is and who is allowed to change it. The table below is a design default, not a rule handed down by any vendor.

The important consequence is that the model never authors the rules it is judged by. If a draft keeps missing a constraint, the fix is a ledger change with an owner, not a longer system prompt. That keeps the constraint set small enough to audit and keeps authorship traceable.

3. Step 1: write the ledger before generating anything

The ledger stores only rules you can state in one sentence and, ideally, detect in one check. Each entry carries an identifier so review comments can quote it instead of re-explaining the intent.

# docs/ledger.yaml — illustrative schema, adapt the fields to your tooling
version: 1
rules:
  - id: no-release-promises
    kind: banned_phrase
    pattern: "(?i)(will ship|guaranteed by|available in Q[1-4])"
    remediation: "State only what the current README or changelog documents."
  - id: cite-source-per-claim
    kind: required_field
    field: source
    applies_to: "feature descriptions"
    remediation: "Link the file, tag, or page the sentence is derived from."
  - id: no-unlinked-numbers
    kind: banned_phrase
    pattern: "(?i)\\b\\d+(\\.\\d+)?%\\b"
    remediation: "Numbers require a linked source path, or they are removed."
human_only:
  - id: security-posture
    applies_to: "any sentence about data handling or isolation"
  - id: deprecation-timeline
    applies_to: "any removal date or support window"

The human_only block is the part most teams skip, and it is the part that prevents the worst failure mode: fluent, plausible sentences about support commitments that no engineer was authorized to make.

4. Step 2: check mechanically whatever is mechanical

A checker does not need to understand prose to be useful, because a large share of rejections are pattern-level. The script below is illustrative and unexecuted; it only implements the banned_phrase rule kind and exits non-zero on a hit.

# check_draft.py — illustrative, not executed for this article
import pathlib
import re
import sys

import yaml


def load_rules(path="docs/ledger.yaml"):
    data = yaml.safe_load(pathlib.Path(path).read_text(encoding="utf-8"))
    return [r for r in data["rules"] if r["kind"] == "banned_phrase"]


def scan(text, rules):
    hits = []
    for rule in rules:
        for match in re.finditer(rule["pattern"], text):
            hits.append((rule["id"], match.group(0)))
    return hits


def main(targets):
    rules = load_rules()
    failed = False
    for target in targets:
        path = pathlib.Path(target)
        for rule_id, text in scan(path.read_text(encoding="utf-8"), rules):
            failed = True
            print(f"{path}: {rule_id}: {text!r}")
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

Wire it into whatever already runs on pull requests, for example python check_draft.py docs/guide.md, and confirm the exit code by running it against a file that deliberately contains a pattern. Report the rule identifier in the failure output so the reviewer can look up the remediation text instead of guessing.

5. Step 3: assemble the prompt from the ledger, not from memory

When the constraint set is a file, prompt construction becomes a lookup rather than a writing exercise. Include the identifiers verbatim and ask the draft to name the identifiers it believes it satisfied, so reviewers have a map to check against.

You are drafting one section of developer documentation.
Allowed: describe behavior documented in the files listed under SOURCES.
Forbidden: availability dates, support windows, security guarantees.
For every feature sentence, attach a source path.
Forbidden rule ids: no-release-promises, no-unlinked-numbers.
Return the body, then a coverage list of rule ids you believe you met.
SOURCES:
- README.md
- docs/api.md

The coverage list is a claim about the draft, not proof of it, and reviewers should treat it that way. Its value is diagnostic: a draft that lists no identifiers usually ignored the ledger, which is cheap to catch before reading the prose. Keep the human-only topics out of the prompt entirely and add them through the attestation block in the next step.

6. Step 4: route review by claim class

Splitting the review queue by claim class stops reviewers from spending attention on text a script can reject. Mechanical violations go back to the author or the generator, factual source links get spot-checked against the cited file, and human-only claims require a named person.

A short attestation block in the pull request template makes the third queue auditable without asking reviewers to remember a policy:

## Human attestation (required for human_only topics)
- [ ] No security or data-handling claims beyond README.md
- [ ] No deprecation or support dates are stated
Attested by: <name> <date>

7. Where a team workbench fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The pattern above assumes several people run the same generation steps, which is where shared configuration matters more than any individual prompt. The repository at chaitin/MonkeyCode currently describes MonkeyAI as an open-source team AI workbench for office, coding, and design work, with shared team resources, configurable models, MCP integrations, access management, and self-hosting. Check the repository's own README for the current description rather than relying on this summary, and note that README.monkeycode.md documents the historical MonkeyCode AI development platform, which should not be assumed to behave identically.

Mapped onto this workflow, three of those documented ideas matter. Shared team resources are a natural home for the ledger and the glossary, so every member's run starts from the same constraints instead of a personal prompt. MCP integrations are the place to attach a source of truth such as the repository or an internal wiki, configured read-only by whoever administers the workspace. Access management is what lets a team separate the people who trigger drafts from the named humans who own attestation, which is the exact boundary this article argues for.

One caution belongs here rather than in a footnote: self-hosting the workbench does not by itself stop data from reaching external model or tool providers. If your configuration routes content to a third-party model endpoint, that provider's terms apply, so verify the data path before drafting anything sensitive.

8. Limits and who should skip this

• The ledger only covers rules someone can write down; vague quality complaints stay in review and should not be encoded as regexes.
• Pattern checks over-match, so expect false positives in code blocks, changelogs, and quoted error messages, and scope targets accordingly.
• Single-maintainer projects and throwaway prototypes gain little from a second artifact that must itself be maintained.
• Contractual documentation, compliance text, and anything with legal exposure should be human-authored end to end rather than drafted against a ledger.
• Model-reported coverage lists are not verification, and treating them as such quietly moves trust back to the generator.

9. How to tell whether it is working

Pick two pages, draft each of them twice with the ledger in place, and compare the rule identifiers flagged by the checker between runs. A shrinking set of repeated identifiers suggests the constraints are reaching the generator; an unchanged set suggests the prompt assembly ignores the file. Separately, count how many attestation fields appear in a week's pull requests and whether the same person signs them, because a signature that always comes from the reviewer of convenience is not ownership. These are method steps, not results, and the numbers you get will depend entirely on your repository.

If you want to try the pattern with a shared workspace instead of per-developer prompts, start from the repository README and confirm which product and version its sections describe before you commit a workflow to it.