幻境地图：幻觉分类

The billing page had the bad news first, and that ordering still bothers me. Logs tell you what happened. The bill tells you what it cost. When the bill gets there first, you have been paying for something for a while already, quietly, without anyone noticing.

Tuesday, 10:14. Our document-ingestion service calls an external API, and wrapped around that call was a retry helper with exactly one rule inside it: if the request does not come back happy, try it three more times. Fixed gap. Same body. Nothing in the helper looked at what had actually gone wrong. Rate-limit responses, server-side failures, a payload with a bad field, a 404 for a resource we had deleted months earlier — every one of them went down the same pipe.

The log lines were almost funny:

POST /v1/documents -> 400 invalid_field: submitted_at (attempt 1 of 3)

That field was already wrong when it left our process. I reproduced it from my laptop with one call against the same endpoint, same body, and got the same 400 with the same field name back immediately, which is what a 400 is for. It would be wrong on attempt two. It would be wrong on attempt three. We were spending quota to ask an API a question we already knew it could not answer.

Behind that call, the backlog kept growing. The daily quota graph flattened out before late morning. A batch that should have finished in about ninety seconds was taking closer to four minutes, and on some days it never really finished at all. Every dashboard we owned stayed green through all of it, because the dashboards counted attempts, and we were producing plenty of those. I put the billing page and the request log side by side on two monitors, and that is how you find the things that are embarrassing because they are obvious. A 404 does not wake up as a 200. A typo does not heal because you asked again politely.

We changed the retry policy that afternoon. The old helper was a loop and a sleep: up to four tries, a flat two-second wait, no branching on the response. The replacement is a small predicate that reads the status code and the headers before deciding anything. Rate-limit responses and server-side errors get retried, Retry-After honored when the response sends one, attempts capped. Everything else returns to the caller with its original error body on the first attempt. No second try, no third, no slot sitting in the backlog pretending it might still succeed. Same service, same traffic, no architecture rewrite, no new dependency.

That Retry-After part deserves its own sentence. We had been sleeping a flat two seconds at a service that sometimes came back with Retry-After: 45 — the API politely asking for forty-five seconds and us answering by knocking again after two. Honoring it took one line and cut our rate-limit retries down further.

The batch settled back to about ninety seconds and the quota survived into the evening. The change itself landed in an afternoon, and most of that was rereading the helper twice to be sure I was not about to remove the one retry that legitimately needed to exist. Understanding the problem is the part I would like back.

For a while I filed all of that under error taxonomy and left it there. It took me longer than I want to admit to notice the same shape hiding inside every hallucination-mitigation plan I have written since. Something goes wrong, we run the thing again, and the fact that we ran it again gets recorded somewhere as a control. This is the corner of the work I sit in — safety and ethics on AI systems — and most of it turns out to be the same question I should have asked about that retry helper: is this a failure that can resolve itself?

The policy bot had been live for three weeks when it drafted a control mapping citing Article 17 and a thirty-day retention limit. Both looked right. The reviewer had the approval one click away. Then somebody clicked the source link instead, and there was nothing there. The thirty-day figure traced back to draft guidance withdrawn more than a year earlier, and the citation pointed at a regulation that does not contain an Article 17 at all. Nothing about the output looked wrong, and that was the dangerous part: errors wearing the costume of compliance.

For a long time our answer to a wrong output was to re-prompt, re-retrieve, re-run. The model-scale version of retrying a 400. Every so often a re-run produced something less obviously broken and we mistook that for a fix. What we were actually doing was hoping, with extra steps and an invoice attached. Wrong outputs are failures of evidence, of time, of scope, or of action, and those four need different instruments. Treat them as one problem and the remedy for one becomes the failure mode of another.

Evidence came first. A phantom citation is a generated reference to a law, a clause, a standard, or an internal policy that either does not exist or does not say what the model claims it says. Our first check asked whether the source existed, full stop. That caught the obvious fakes. What it missed was a real source attached to a claim it does not support: a genuine clause about data subject rights cited in support of a claim about breach notification, or an Article 17 pulled from a regulation that does have one, just not the one we operate under. The source is real. The mapping is wrong, and an existence check cannot see the difference.

Now every normative claim carries a source URI, a retrieval timestamp, and an exact quoted span. If the span cannot be located in the retrieved text, the claim gets flagged rather than thrown out, because sometimes the model paraphrases correctly and the mismatch is a formatting artifact. Flag rates are high enough that throughput drops and the backlog of things waiting on a person grows. That is the trade, and we took it deliberately: strict span checks make fluent summaries harder to pass.

Dates are subtler. We once applied a retention requirement that was correct in 2021 to work done after the rule changed in 2023. The model cited the old value because that is what dominated its training data. First attempt at a fix: store the effective date next to the source. Simple, until you remember that amendments form intervals — valid from one date, replaced by another interval, which may itself have been superseded. Every source needed effective_from, effective_to, and retrieved_at. The check itself is trivial: reject any claim whose validity window does not contain today. Keeping those windows current by hand is not. Every amendment means closing one interval and opening another, and the one you miss brings the phantom back wearing a date field that makes it look more trustworthy than it did before.

Scope fails quietly too. Same rule, wrong jurisdiction. We mapped a retention rule from one regulatory regime onto an entity operating under a different one. Rule real, citation correct, application wrong. Our first pass did string matching on entity names, and "Acme Corp" matched "Acme Holdings" because the prefix was close enough. That rejected valid mappings and waved through bad ones. Now entities resolve against a controlled registry before any rule applies, and each retrieval query carries jurisdiction and scope tags. The cost is false negatives: a parent/subsidiary relationship that should qualify gets held back because the registry does not encode it. We are still tuning that, and we biased it toward caution on purpose.

Numbers fail in their own small way. Wrong retention periods, wrong deadlines, percentage caps that quietly double. The model reads "up to 30 days" and outputs "30 days" as a hard limit, dropping the qualifier that carried most of the meaning. The first version extracted numbers as free text and compared them as strings, so "30 days", "30", and "thirty days" all looked different to the checker. Now numbers come out as typed fields with units, compared against authoritative tables, and free text never sets a control value. The parser still trips on phrasing with no number in it. "Within a reasonable period" has nothing to extract, and we will not guess. Those go to a person.

Procedures fail differently again. A workflow step invented, omitted, or ordered wrong — the model claiming a review is required before submission when the policy requires it after. We tried representing procedures as state machines and diffing generated steps against the approved graph. Policy exceptions are real, though. The graph cannot express "except when the data class is X and the jurisdiction is Y", so it held back procedures that were correct. We still run the diff, but it is advisory: it flags the step and a reviewer decides. Slower. The alternative was enforcing a model of the procedure that is less accurate than the procedure itself.

And omissions, the one that scares me. A required control, a disclosure, an escalation path, simply missing. Nothing looks wrong, because nothing looks wrong. The output is coherent, just incomplete. Our first coverage check used a checklist derived from the same draft that produced the mapping. Circular: the draft had the omission, the checklist had the omission, the mapping passed clean. Now the checklist comes from the regulation or standard itself and never from anything the model generated. Checklists drift from source text as interpretations evolve, so we version them behind the same validity gate as the rules they cover. Tedious. Also the only way I have found to keep an omission from being invisible.

We tried full automation once. Every claim span-checked, every number typed, every procedure diffed. The backlog grew faster than throughput, and the compliance folks said plainly that they did not want a system that is 100% correct and 30% useful. They wanted the dangerous things caught and the rest left to people. So we landed in the middle: high-confidence claims pass, low-confidence claims get flagged, and anything touching numbers, dates, or jurisdiction gets a second pass. We are still calibrating where that line sits. It has not been validated at scale. It might not hold. I would rather say that out loud than ship a dashboard implying otherwise.

What I keep coming back to has more to do with authority than with accuracy. The model may propose a mapping. It does not get to be the authority on whether the mapping is true. Everything above — the span check, the validity window, the registry, the typed numbers, the procedure diff — is machinery for keeping that boundary intact when the output is fluent enough that everyone wants to skip it. The machinery is only worth building if a person still owns the line. The checkers matter less than the signature.

Which is where the retry story comes back around. We retried everything because we never asked whether a given failure could resolve itself. We re-prompt everything for exactly the same reason. A phantom citation is not a condition that clears if you give it a minute; no amount of re-running will make Article 17 exist. The illusion is that motion is progress, and the invoice is the only thing that ever disagrees.

So before you re-prompt, re-retrieve, or escalate: ask whether this particular failure can fix itself. Evidence failures need a source. Time failures need a validity window. Scope failures need entity resolution. Action failures need a procedure diff. Sort first, then pick the instrument. Some of these will clear on their own. Most will not, and those are the ones where another attempt just burns quota you will want later.

I still think about that bot, three weeks in, one click away from approving a control built on a rule that never existed. What stayed with me was how good it looked. Next time a phantom shows up, I will do what we should have done with those retries: open the dashboard, open the logs, and work out whether this one heals on its own before I try again.