Your AI-written README is probably lying: an evidence-first docs loop (with 2 free prompts)

Ask Cursor, Claude Code or ChatGPT to “write a README for this repo,” and you’ll get something that looks exactly like a README: a features list, an installation section, a quickstart, usage examples, a contributing section, maybe a few badges. Then a new teammate runs the quickstart, and step two fails, because the script is called dev, not start, the --verbose flag it documents never existed, and the .env.example it tells them to copy isn’t in the repo.

The model didn’t malfunction. It did what it was asked: produce a finished document. When it hit a gap, it filled it with what projects like yours usually have. Fluent prose makes the guesses look exactly like the facts. Here’s the loop I use instead, and two full prompts you can paste today.

Why “write docs for this” goes wrong

The failure modes are predictable:

• Convention filling. npm start, --config, DATABASE_URL, LOG_LEVEL, a 404 on every endpoint. Each one is plausible, and any of them can be false for your code.
• Invented rationale. Ask for comments that explain why, and you get reasons (“to avoid rate limiting”, “for performance”) that nobody ever gave. The next maintainer believes them.
• Untested commands. Setup and runbook steps nobody ran look identical to ones somebody did.
• Confident numbers. Defaults, limits, timeouts, supported versions. The most-copied parts of a doc are also the easiest to make up.
• Write-once. Even a perfect doc starts drifting at the next merge.

None of this needs a smarter model. It needs a process where the model has to show its sources, and where “I don’t know” has somewhere to go.

The loop: evidence → outline → draft → verify → keep in sync

evidence → outline → draft → verify → keep in sync

1. Evidence. Before any prose: who is the reader, and what facts will the doc rely on, each with a source (file:line, commit, PR, note)? Whatever the evidence doesn’t cover becomes a question.
2. Outline. Sections mapped to evidence rows. A section with no evidence is dropped or kept as a visible TODO stub. Cutting a section here costs nothing.
3. Draft. Prose constrained by the outline. Commands copied from scripts, not “improved”. Unknowns written as TODO: <question>.
4. Verify. Run the copy-paste path on a clean machine. Grep every identifier. Check every number and every “because”.
5. Keep in sync. Audit the docs against the code before each release, and after every incident for runbooks.

If you only take one rule from this post: a visible TODO beats a plausible guess. Tell the model that explicitly. Models are trained to be helpful, and “helpful” defaults to filling the gap. Saying that the gap is the helpful answer changes what you get back.

Free prompt 1: README from evidence, not from vibes

Paste this with the repo @-mentioned (Cursor), readable (Claude Code), or with the file tree, manifests and scripts pasted in (ChatGPT / Copilot Chat).

You are a senior engineer writing a README for this repository. You write only what the
code, config and files I give you can support. Anything you cannot support from evidence
becomes a visible TODO, not a guess.

## Context
- Repository / paths to read: {{REPO_PATHS}}
- What the project is for, in my words (may be incomplete): {{PROJECT_PURPOSE}}
- Who reads this README: {{AUDIENCE}}
- Commands I know work, or "unknown": {{KNOWN_COMMANDS}}

## Process (plan before writing)
1. Evidence scan. Read manifests, scripts (Makefile, package scripts, CI workflows),
   entrypoints, config and env var reads, LICENSE and existing docs. Record each fact
   you will rely on with its source file:line.
2. Gaps. List what a reader needs that the evidence does not tell you. Each gap becomes
   a TODO with a question for the maintainer.
3. Outline. Propose sections; for each, name the evidence rows it uses.
   STOP here and wait for my approval.
4. Draft. Quickstart in clean-clone order: prerequisites, install, configure, run, test.
   Copy commands exactly as defined in scripts.
5. Verification list. Every command, path, env var and flag in the README with its
   source, marked RAN (only if you executed it successfully) or UNVERIFIED.

## Output
### Evidence table   | # | Fact | Source (file:line) | Used in section |
### Gaps & TODOs
### Proposed outline
### README.md (after approval)
### Verification list | Item | Source | RAN / UNVERIFIED |

## Rules
- Never invent APIs, CLI flags, env vars, config keys, commands, defaults or behaviour
  not in the code. Mark unknowns as TODO: <question>.
- No badges, registries or CI services the repo doesn't configure.
- No "blazing fast" / "production-ready" without evidence.
- The license line must match the LICENSE file; if there is none, write TODO.
- Short and correct beats long and plausible.

Three parts do most of the work:

• The evidence table. Every section has to point at rows. A Docker section with no Dockerfile has nothing to point at.
• The stop after the outline. You review ten lines of structure instead of three pages of prose.
• RAN / UNVERIFIED. In a chat UI, everything is UNVERIFIED, and that’s honest. You run the quickstart; then you flip the labels.

Expect a few TODOs. A README written from incomplete material with zero TODOs is the one to distrust.

Free prompt 2: the doc drift audit

Docs were right once. Since then, a flag was renamed, a default changed, and a new required env var appeared. This prompt turns “the docs look mostly fine” into a ledger you can act on.

You are a meticulous technical reviewer auditing documentation against the current code.
You treat every concrete statement in the docs as a claim to verify. The code is the
source of truth; you never change code to match the docs.

## Context
- Docs to audit: {{DOC_PATHS}}
- Code to verify against: {{CODE_PATHS}}
- What changed since the docs were last correct (git log / diff --stat, or "unknown"):
  {{CHANGE_SUMMARY}}
- Who reads these docs and what they do with them: {{READER}}

## Process
1. Extract every checkable claim and give it an ID: commands, CLI flags, env vars,
   config keys, defaults, endpoints, file paths, function names, versions, ports,
   limits, behaviour statements ("retries three times").
2. Verify each against the code:
   MATCH (cite line) / DRIFT (code says X, doc says Y; cite both) /
   MISSING-IN-CODE (list the search terms you used) /
   UNDOCUMENTED (user-facing things in code no doc mentions) /
   UNVERIFIABLE (say what would settle it).
3. Rank by reader harm: HIGH = copy-paste fails, data loss, security, wrong limits;
   MEDIUM = wrong for an edge case; LOW = wording. Add confidence per verdict.
4. Report and STOP. Do not edit anything.

## Output
### Summary (counts per verdict, top five HIGH-harm items)
### Claim ledger | ID | Doc location | Claim | Verdict | Evidence / search terms | Harm | Confidence |
### Undocumented user-facing changes
### Prevention (1–3 suggestions using tooling the repo already has)

## Rules
- Never invent APIs, flags, env vars or behaviour.
- If the doc describes behaviour the code lacks, report DRIFT and ask which is right.
- UNVERIFIABLE is an acceptable verdict. Don't upgrade it to MATCH.
- Mark anything you're unsure about as TODO: verify <question>.

Two details matter:

• It checks both directions. Stale claims in the docs are half of drift. The other half is the new required env var nobody wrote down. That’s the UNDOCUMENTED section.
• “Code is the source of truth” is a rule, not a vibe. Without it, an agent with write access may “helpfully” re-add a removed flag so the README is correct again.

Run it right before a release on the docs users touch first: the README quickstart, the install page, the config reference. Paste the release’s git diff --stat as the change summary, so the model knows where drift is most likely.

Why-comments: the rule that saves the next maintainer

Comments that narrate code (# loop over users) are noise. Comments that explain why are gold, and also where models invent the most. The fix is a simple constraint:

A why-comment must come from the code itself, a ticket, a commit message or a person. Otherwise write TODO(why): <question>.

Then read the list of TODO(why) questions the model produces. It’s often the most valuable output of the session: a list of exactly the knowledge that disappears when someone leaves. Paste it into the PR and let the author answer.

The 10-minute hallucination check

Before you merge any AI-written doc:

1. Pull out every command, flag, env var, config key, endpoint and file path.
2. Search each one in the repo (git grep -n "<identifier>"). Anything not found is guilty until proven innocent.
3. Run the copy-paste path on a clean environment, in order.
4. Every number (default, limit, port, version) needs a source line.
5. Every “because” needs a source. No source, no because.

And don’t let AI write these alone: security or compliance statements, SLAs, license text, incident disclosures, or destructive-operation steps. A human owns every sentence there.

The shape that works

A column titled “Source (file:line)” has to be filled per row, and an empty cell is visible in a way a confident paragraph never is. That’s the whole trick.

Try it on a doc you know well: regenerate your own README with prompt 1 from the code alone, or run prompt 2 against it, and check every claim and every TODO it raises.

I packaged the eight documentation prompts I use (README from a codebase, why-comments & docstrings, API reference / OpenAPI, ADRs, changelog & release notes, onboarding guide, runbooks, doc drift audit) plus two checklists, a Cursor rule and a PDF guide, as the AI Documentation Prompt Pack: https://payhip.com/b/a8p1N. Everything above works without it.

— Pluto · @Mars_Peakul