Separate Owned Error Rows From Drafted Support Summaries
Support copy stays trustworthy only when the error contract is frozen before any model writes a sentence. A draft may explain a registered code, but it may not invent a status, a retry rule, or a recovery path. The worki
Support copy stays trustworthy only when the error contract is frozen before any model writes a sentence. A draft may explain a registered code, but it may not invent a status, a retry rule, or a recovery path. The working control is a human-owned catalog plus a checker that rejects any prose touching owned fields. This article specifies that split, a local Python checker, and the situations where the split is the wrong control.
The failure mode in generated support pages
Generated support pages often sound complete while the contract underneath has already drifted from the service. A model may fill a missing retry hint, rename a code for readability, or borrow a status from a neighboring endpoint. Readers then follow a recovery path the service does not implement, and later tickets cite the page as authority. The remedy is not a softer prompt, because a frozen catalog row must be cited and must not be rewritten.
Split each page into owned rows and drafted sentences
Treat each public error as a row that a human edits, reviews, and versions before any narrative is requested. The row holds the code, the HTTP status, the retry class, and the stable identifier that prose is allowed to mention. A model may draft the surrounding explanation, a short user-facing summary, and a non-executable illustration of the message. It may not add fields, drop fields, or restate owned values in words that a later editor could mistake for a new contract.
What a human must own
A human owns every value that would change client behavior if it were wrong, even when the sentence looks harmless. That set includes the public code, the transport status, the retry class, the idempotency note, and the catalog version. A human also owns the decision that a code is public, because unpublished codes must not appear in drafted support copy. Ownership here means a reviewed file in the repository, not a comment in a chat transcript that nobody can diff.
What the model may draft
The model may draft a plain-language cause, a reader next step that quotes the catalog identifier, and a short caution. Those sentences must point at a registered identifier and must avoid numerals that look like statuses or retry counts. Examples may show the shape of a message only when the example is marked non-executable and contains no new codes. If a sentence needs a status, a wait interval, or a code, the writer inserts a catalog citation instead of free text.
Gate each atom before you accept a draft
Use the table below as the review gate before a draft is accepted into the support section of the docs site. An atom is owned when changing it would change what a client sends, waits for, or treats as success. An atom is draftable when deleting it would leave the contract intact and would only reduce explanation quality. Mixed atoms, such as a sentence that both explains and states a status, must be split before the checker runs.
| Atom | Owner | Model may draft | Checker rule |
|---|---|---|---|
| Public error code | Human | No | Exact citation match |
| HTTP status | Human | No | Absent from draft prose |
| Retry class | Human | No | Citation only, no timing paraphrase |
| Idempotency note | Human | No | Must not appear as free text |
| Catalog version | Human | No | Draft front matter must match |
| Cause explanation | Model, then human review | Yes | Must cite a registered id |
| User summary | Model, then human review | Yes | No bare status numerals |
| Non-executable illustration | Model, then human review | Yes | Flag false, and no new codes |
A failing summary states a status and a wait interval in free text, which turns an explanation into an unofficial contract. A passing summary cites the registered identifier and describes the cause without numerals that a client could treat as protocol. Keep both samples beside the checker so reviewers can see the rule without reading the regular expressions first. The samples are fixtures for review, not transcripts captured from a live service, so do not publish them as runtime evidence.
Failing: HTTP 429 means retry after 30 seconds under ERR_RATE_LIMITED.
Passing: See catalog row ERR_RATE_LIMITED for status and retry class. The call was rejected because the registered limit was reached.
Workflow
1. Freeze the catalog
Start from the service's published error list and copy only the fields a client needs in order to behave correctly. Store that extract as YAML in the docs repository, and require a human review whenever a row is added or changed. Do not ask a model to generate this file, because the file is the source the later draft is judged against. Record a catalog version in the file so a draft cannot silently describe an older or newer contract.
2. Request a narrow draft
Ask the model for explanations only, and pass the registered identifiers as the sole list of codes it may mention. Instruct it to emit YAML with a citation field, a summary field, and an optional illustration marked non-executable. Tell it to omit statuses, retry intervals, and any code that is not in the identifier list you supplied. Keep the prompt with the pull request so a reviewer can see the constraint, not only the resulting sentences.
3. Run the checker before review
Run the checker in the same environment you use for docs continuous integration, and fail the job on any violation. The checker compares citations to the catalog, scans summaries for bare status patterns, and rejects unknown code tokens. A passing run is not editorial approval, because tone, accuracy of cause, and product fit still need a human reader. Treat checker output as a defect list, and do not patch owned rows to make a fluent draft pass.
4. Review the explanation, not the contract
Reviewers read the summary for clarity and for claims that the catalog does not support, such as implied guarantees. If the cause is wrong, edit the draft or send it back, and leave the catalog row untouched unless the service changed. If the service changed, update the catalog in a separate commit so the contract diff is visible without narrative noise. Publish only after both the checker and the human review have passed on the same catalog version.
A proposed checker and two fixtures
The script below is a proposed local checker, and it has not been presented here as a benchmark or as a production incident report. It expects Python 3.9 or newer, PyYAML, and a catalog that a human has already reviewed in version control. Exit code zero means the draft stayed inside the citation rules, not that the explanation is accurate or ready to publish. Install the dependency in the docs environment you already trust, and do not treat a missing parser as a passing result.
The version string 2026-10-13.1 is an example pin for these fixtures, not a vendor release identifier. Replace it with the version your reviewers already use for the service error list.
# catalog/errors.yaml — human-owned. Do not generate this file.
catalog_version: "2026-10-13.1"
errors:
- id: ERR_RATE_LIMITED
http_status: 429
retry_class: retry_after_header
idempotency: safe_to_retry
public: true
- id: ERR_CONFLICT
http_status: 409
retry_class: do_not_retry
idempotency: unsafe_to_retry
public: true
# drafts/support.yaml — model may draft summary and illustration only.
catalog_version: "2026-10-13.1"
entries:
- citation: ERR_RATE_LIMITED
summary: >
The service rejected the call because the registered limit for
this identifier was reached. Follow the catalog row rather than
guessing a delay.
illustration:
executable: false
body: "error_id=ERR_RATE_LIMITED"
"""Proposed checker. Not a measured benchmark. Run it locally before review."""
from __future__ import annotations
import re
import sys
from pathlib import Path
import yaml
STATUS_RE = re.compile(r"\b(?:[1-5]\d{2})\b")
CODE_RE = re.compile(r"\bERR_[A-Z0-9_]+\b")
RETRY_HINT_RE = re.compile(
r"\b(?:retry after|wait \d+|backoff|milliseconds|seconds)\b",
re.IGNORECASE,
)
def load(path: str) -> dict:
data = yaml.safe_load(Path(path).read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise SystemExit(f"{path}: expected a YAML mapping")
return data
def check(catalog: dict, draft: dict) -> list[str]:
found = []
if draft.get("catalog_version") != catalog.get("catalog_version"):
found.append("catalog_version mismatch")
owned = {
row["id"]: row
for row in catalog.get("errors", [])
if isinstance(row, dict) and row.get("public") is True and "id" in row
}
for entry in draft.get("entries", []):
if not isinstance(entry, dict):
found.append("entry is not a mapping")
continue
citation = entry.get("citation")
summary = entry.get("summary") or ""
if citation not in owned:
found.append(f"unregistered citation: {citation}")
continue
status = str(owned[citation].get("http_status", ""))
if status and re.search(rf"\b{re.escape(status)}\b", summary):
found.append(f"{citation}: owned status restated in summary")
if STATUS_RE.search(summary):
found.append(f"{citation}: bare status in summary")
if RETRY_HINT_RE.search(summary):
found.append(f"{citation}: retry timing leaked into summary")
for token in CODE_RE.findall(summary):
if token != citation:
found.append(f"{citation}: foreign code {token}")
illustration = entry.get("illustration")
if illustration is None:
continue
if not isinstance(illustration, dict):
found.append(f"{citation}: illustration must be a mapping")
continue
if illustration.get("executable") is not False:
found.append(f"{citation}: illustration must be non-executable")
body = illustration.get("body") or ""
if status and re.search(rf"\b{re.escape(status)}\b", body):
found.append(f"{citation}: status leaked into illustration")
for token in CODE_RE.findall(body):
if token != citation:
found.append(f"{citation}: foreign code in illustration {token}")
return found
def main() -> int:
if len(sys.argv) != 3:
print("usage: check_support_draft.py CATALOG.yaml DRAFT.yaml")
return 2
problems = check(load(sys.argv[1]), load(sys.argv[2]))
if problems:
print("\n".join(problems))
return 1
print("catalog citations accepted; human review still required")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Pass the catalog path first and the draft path second, and treat a non-zero exit as a blocked documentation change. Print the violation list into the pull request log so the author sees the rule name rather than a generic failure. Do not auto-rewrite the summary inside the checker, because a silent edit would hide the contract leak from the reviewer. Keep the checker free of network calls so a documentation job cannot depend on a live service or a remote model.
python3 -m pip install pyyaml
python3 check_support_draft.py catalog/errors.yaml drafts/support.yaml
echo "exit=$?"
How to read a rejection
A version mismatch means the draft describes a different contract, so discard it until a human updates the catalog. A bare status means the model restated an owned field, so delete the numeral and leave a citation in its place. A foreign code means the draft introduced an identifier the catalog does not list, which is a contract change in disguise. An executable illustration means a reader might run sample text that was never captured from the service, so mark it false or remove it.
Run the draft loop without moving ownership
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode's free model access and free server option can host this draft-and-check loop without changing who owns the catalog rows. A free model pass is enough for the narrow draft, because the checker, not the model, decides whether a citation is allowed. A free server option is a reasonable place to run that pass beside the checker, provided you keep the catalog in your own repository. Availability can change, so treat both options as a convenience for the current drafting session, not as a capacity plan.
The catalog and the draft should travel with the pull request, because a remote session is only a place to generate text. Do not store unpublished codes in a remote prompt if your policy forbids sending internal identifiers outside the repository. If either option is unavailable, the same checker still runs on any machine that has Python and the two YAML files. If you already keep a human-owned error list, try the checker on one support page before you widen the draft scope.
Reproduce these checks before you trust the script
The following checks are a specification for the checker, not a report of production traffic or measured model accuracy. Copy the catalog fixture, then confirm that a matching citation and a non-executable illustration produce no violations. Change the draft version string and confirm the checker exits non-zero with a catalog version mismatch.
Insert a three-digit status in the summary and confirm the bare-status rule fires even when the citation is valid. Add a second error token that is absent from the catalog and confirm the foreign-code rule rejects the entry. Set the illustration executable flag to true and confirm the checker refuses the file before any reviewer reads the prose.
# failing fixture — expected violations, not a recommended page
catalog_version: "2026-10-13.1"
entries:
- citation: ERR_RATE_LIMITED
summary: "HTTP 429 means retry after 30 seconds."
illustration:
executable: true
body: "ERR_UNKNOWN"
Limitations of the checker
The checker does not know whether a cause explanation is true, because truth still depends on service behavior and a human reader. It will miss a wrong wait instruction written as words without digits, so the retry-hint pattern is a backstop rather than a proof. It will also miss a status written as a word, such as a spelled-out client error, unless you extend the banned phrase list.
YAML shape errors, encoding mistakes, and missing keys need ordinary parser handling, which this sketch leaves to the caller. The method does not replace contract tests against the running service, and a green docs check can still describe a stale deployment. The status pattern also flags incidental three-digit numbers, so spell counts or move them outside the drafted summary.
Who should not use this split
Skip this split if you do not yet have a human-owned list of public errors, because the checker would only enforce an empty file. Skip it for conceptual overviews and launch announcements, where the risk is unsourced claims rather than client-visible error codes. Skip it when the documentation is the legal contract itself, because a prose checker is not a substitute for counsel or a signed specification. Teams that cannot review drafts should not add a model, since the checker only blocks a narrow class of leaks.
Keep the contract ahead of the summary
Freeze the error catalog, let the model draft only cited explanations, and reject any sentence that restates an owned field. That order keeps support copy useful without letting a fluent draft become the contract your clients actually follow. Run the fixture checks before you trust the script in continuous integration, and keep human review on every accepted summary. The catalog remains the contract, the summary remains an explanation, and the checker remains a gate rather than an author.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.