Dev.to AI 🤖 Ai 👁 0 📖 2 min read

I built a RAG that refuses to answer — and measured when it should

Most RAG demos share a failure mode nobody demos: they always answer. On a corpus of medical evidence, that stops being a UX quirk and becomes a dangerous one. I built a dietary-supplements evidence RAG over 17,698 evid

Most RAG demos share a failure mode nobody demos: they always answer. On a
corpus of medical evidence, that stops being a UX quirk and becomes a
dangerous one.

I built a dietary-supplements evidence RAG over 17,698 evidence-grade
PubMed articles. It answers only when the retrieved studies support an
answer, stamps every claim with the PMID it came from, and abstains at the
corpus boundary instead of guessing.

The part worth writing about isn't the pipeline — it's how I know the
refusals are right.

The system, in one glance

gate → hybrid retrieve (BM25 + dense, RRF) → answer contract
     → deterministic verify + judge → one repair pass
     → answer with citations, or abstain

Extractive-first: the default lane quotes study text instead of generating
prose. Every run persists its full trace — gate decision, retrieved chunks,
per-claim stamps, verdicts — at a public URL anyone can inspect.

The eval contract

Two question sets, both committed to the repo.

golden.jsonl — 126 stratified questions. This is what I iterate against.

heldout.jsonl — 199 disjoint questions. This one only measures, never
tunes. When golden and held-out disagree, I've overfit.

The questions come from real sources (SuppLLMent effectiveness items,
MedQuAD consumer questions), with one admission rule: a question enters the
pool only if BM25 actually retrieves evidence for it. 4,026 made it in;
1,543 were rejected — each with the reason kept in rejected.jsonl. A
question nobody can answer in the corpus isn't a test, it's noise.

What the measurement found

On the golden run, every abstention was correct — zero dangerous answers
slipped through. That's the headline.

The less comfortable finding: about 45% of answerable rows abstained too.
Partly the verifier being too strict, partly non-deterministic generation.
So I added a rescue pass — when the system abstains but evidence sits at
the top of the pack, it re-asks whether that evidence actually answers the
question. Advice-decline rows went from 20% to 36% rescued; answerable rows
stayed around 22%, because some of them legitimately lack coverage.

That's the real shape of this work. Less "make the model better," more:
measure a failure class, find the root cause, fix it, re-measure on a set
you didn't tune against.

Try it, check it

The live demo runs on my site — ask it about a supplement and watch it
cite real studies, or watch it refuse. The repo and both eval sets are
public.

I'm an AI engineer in Colombia working remotely (UTC-5). If you're building
systems where "I don't know" is a feature, I'd genuinely like to compare
notes.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.