How to Stop an AI Assistant From Inventing Numbers in Business Reports

A method, not a guarantee: no setup removes errors completely. We give no accuracy percentages because we have not published a measurement.

Why does an AI assistant invent numbers?

A language model writes text that sounds right. If you ask for a figure that is not in front of it, it can produce a plausible one. The risk is highest when the model is asked to remember, estimate or sum figures from a long document. The fix is not a better prompt alone; it is a different division of labour.

What is the basic rule?

Numbers come from the data, calculations come from code, and the model writes the explanation. The assistant should never be the source of a figure. If you keep that rule, most invented numbers disappear because the model has nothing to invent: it is handed the figures it may quote.

Control 1: how do I show where each number comes from?

Attach to every figure its origin: the table, the cell or the query, and the date. In a report this can be a footnote or a hover. A number without an origin is treated as unverified and should not be published. This also lets a reader check a figure in seconds.

Control 2: why should totals be calculated outside the model?

Sums, averages, percentages and year-on-year changes are deterministic. A spreadsheet formula or a database query gets them right every time; a language model sometimes does not. Compute them first, then pass the results to the model as given facts.

Control 3: what should the assistant do when a figure is missing?

Write "data not available" and say which source was empty. Tell the assistant explicitly that this is the correct answer when data is absent. Without that instruction, models tend to fill the gap.

Control 4: how do I catch figures that look wrong?

Compare each figure to the previous period and to a plausible range. A cost line that moves by an order of magnitude, or a margin above 100%, is flagged automatically. These checks are cheap and catch both data errors and model errors.

Control 5: who reviews, and what do they review?

A person reviews only the figures that were flagged or that changed the conclusion, not the whole report. Keep the reviewer's name and date on the report. If no figure is flagged, say so in the report.

What can I measure to know the setup works?

Count the errors found by the controls and by readers over a period, compared with the number of figures checked. Publish the method with the number, or do not publish the number. We do not publish an accuracy figure for S.C.A.L.A. today.

What should I ask any AI vendor, including S.C.A.L.A.?

Ask to see the origin of any number you plan to use, and ask what the assistant does when data is missing. If the answer to either is "the model estimates it", do not use the figure in a report.

Originally published at https://get-scala.com/en/blog/how-to-stop-ai-inventing-numbers-in-business-reports.