How to check your AI's data answers: evals for each stage

TL;DR. Your assistant gives a number, and you cannot tell if it is right. An eval is a repeatable test of its answers against something you trust, and no single eval is enough. Trusted queries show that something broke, checks without an answer key catch errors on every answer, and a weekly sample finds what you never tested. An AI judge scales the review once people have checked it.

Start with trusted queries before launch, and add layers as money and askers grow.

Why is one test file not enough?

The previous post argued that judging answers is the new data work once AI makes answers cheap. This post shows how to do that judging without reading every query.

Here is how I think about it. Each kind of eval catches one failure and misses others, so you add them in layers as the stakes grow.

The problem is not new, and it is not solved. The BIRD benchmark (2023) tests text-to-SQL, turning a question into a database query, on large and messy real databases. It scores execution accuracy: run both queries and compare the results. In 2023, people scored 92.96 percent and the best model, ChatGPT, scored 40.08 percent. In June 2026, Google Research announced 80.04 percent for its Gemini-SQL2 on the BIRD single-model track, a figure Google reported itself. Models improved a lot, and are still below people on messy data.

What does a fixed set of trusted queries catch?

A fixed set is a short list of real questions, each paired with a query your team approved. The eval runs that query on today's data, runs the assistant's query, and compares the two results.

A frozen number goes wrong the day new invoices arrive. A trusted query stays right as data changes, and needs an update only when a definition changes.

The limit: it covers only the questions you thought of, and the key itself can be wrong. Researchers at the University of Illinois Urbana-Champaign (CIDR 2026) audited public text-to-SQL benchmarks. They found annotation errors in 52.8 percent of BIRD Mini-Dev problems. Among the 121 Spider 2.0-Snow problems with released reference queries, 66.1 percent had errors. After fixing 100 BIRD Dev problems, five agents' scores changed by between 3 percent lower and 31 percent higher, and rankings moved by up to three places. If expert keys go wrong that often, your trusted queries need an owner and a review date.

What can you check without an answer key?

Some rules must hold whatever the right answer is, so they can run on every live answer.

• Same question, different words. Both phrasings give the same number.
• Parts add up. Revenue by region sums to total revenue. Use this for amounts, not for customers who can sit in two groups.
• Sources agree. Billing and your CRM agree within a margin you set in advance.
• The definition exists. The cited definition is in your metric files.
• It says when it cannot answer. Ask something your data cannot answer. It should say so.
• Jumps get flagged. A change from last period beyond a threshold you set per metric goes to a person.

This is metamorphic testing: checking how answers relate, not whether each matches a key. A March 2026 study, "Semantic Invariance in Agentic AI", rephrased multi-step scientific reasoning problems for seven models. Bigger models were not reliably more consistent. The most stable one kept its result nearly unchanged in 79.6 percent of cases. The test used only 19 science problems, not data questions.

The limit: if the assistant always uses the wrong definition, every check passes.

What does a weekly sample of real questions show?

In my view, this is the layer small teams skip and need most. Each week, take a random handful of questions people actually asked. A person who knows the business marks each answer right or wrong, and gives each miss one cause:

• Missing definition. No written meaning existed for the term.
• Unclear instruction. A definition exists, but nothing says to use it.
• Wrong data. It used the wrong or an incomplete table.
• Stale trusted answer. The assistant was right and the key was out of date.

Each miss becomes a new case in the fixed set. This is where new question types show up first.

When can an AI judge do the review?

An AI judge is a second model that grades each answer. For data answers, I think it should answer narrow questions: did the answer use the definition it cites, and does the query fit the question?

In the MT-Bench paper (2023), strong judges agreed with human preferences over 80 percent of the time, but favored the first answer shown, longer answers and their own. A June 2026 study, "Reliability without Validity", tested 21 judges from nine providers across about 541,000 judgments. Plain match rate overstated judge quality: on MT-Bench, agreement beyond chance (kappa) was 33 to 41 points lower. Judges repeated their verdicts reliably on reruns, yet two judges used in production favored answers by position.

So calibrate it on your own answers: people and the judge mark the same set, and you measure agreement beyond chance. The limit: a stable judge is not a correct one. Never let it decide alone on money questions.

What should you add at each stage?

I think the layers should arrive in this order.

Before launch. Use the fixed set only: a handful of real questions, each with a trusted query and the definition behind it. Rephrase the key questions by hand and compare. If your people ask in Hebrew, rephrase in Hebrew too.

First paying customers. Rerun the set automatically on every change to the model, the instructions or a definition, and once a week. Add the checks without an answer key to every money number. Start the weekly sample and the cause list.

Growing with a first data person. Add an AI judge calibrated on human marks. Track the pass rate over time; only a fixed set makes improvement visible. Keep a set of cases you never use while tuning instructions, and fix (pin) the model version, recording it with each run.

In my estimate, not a measured figure, the cost looks like this:

What goes wrong, and how do you catch it?

The trusted answers are wrong. The key is wrong, and the team changes the assistant to match it.
Guardrail: every trusted query has a named owner, a definition and a review date. Write it without the assistant's help, because two queries built on the same wrong definition will agree.
Check: when a case fails, have its owner rerun the trusted query by hand before anyone touches the assistant. A failing money case blocks the change until the owner decides which side is wrong.

The AI judge looks reliable but is not valid. It gives the same verdict every time, yet marks wrong answers as right.
Guardrail: calibrate it against human-marked answers using agreement beyond chance, and keep people on money questions.
Check: give the judge a few answers a person marked wrong. If it passes them, it is not ready.

Answers shift silently after a model change. The provider updates the model and nothing shows an error.
Guardrail: pin the model version, record it with each run, and rerun the full set before switching.
Check: compare this week's run with last week's. A case that changed result while its definition did not needs a person.

For your data team

This part gets technical. One case file, with made-up values:

# evals/paying_customers.yaml
case: q-001
question: "How many paying customers did we have on 30 September 2026?"
trusted_query: queries/paying_customers_at_date.sql   # run on today's data, not a stored number
params: {as_of: 2026-09-30}
definition: metrics/paying_customer.md@v2
owner: finance_lead
last_reviewed: 2026-10-01
compare: result_set          # execution accuracy
holdout: false                # true = never used while tuning
invariants:
  - rephrase: ["How many customers were paying at the end of September?"]
  - parts_sum: {by: region, tolerance_pct: 0.1}   # amounts only
  - cross_source: {other: crm.accounts, max_gap_pct: 2}   # set per metric
  - cites_definition: true
  - jump_vs_prior: {max_change_pct: 25}   # set per metric
run:
  model: pinned-model-2026-09     # recorded with every run
  repeats: 3                      # a case that passes only sometimes is a finding
  triggers: [model_change, instruction_change, definition_change, weekly]

1. Ask the question in a fresh session with read-only credentials and the real instructions.
2. Run the trusted query and the assistant's query, and compare result sets.
3. Run each invariant, including every rephrasing.
4. Diff against the previous run, so new failures stand out.
5. Send misses to the owner with a cause tag, and add or fix a case in the same pull request.

Check it in 10 minutes

1. Pick one money question your team asks often, such as paying customers last month.
2. Write the query you trust, run it, and note its definition.
3. Ask the assistant the question in two phrasings and compare all three numbers.
4. Ask for the same number by region or plan, and check the parts add up.
5. Put an owner's name and a review date on the trusted query.

See where your setup stands with the two minute self-check. Then read the draft checks your coding agent can run.

Originally published at goagentdata.com.