Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 2 min read

Turning "Check If a Citation Is Real" Into a Browser Extension With No Backend

LLMs are very good at producing references that look right. They're well-formatted, topical and often carry a DOI that resolves. The catch is that the DOI frequently resolves to a different paper. In one benchmark, 99 of

LLMs are very good at producing references that look right. They're well-formatted, topical and often carry a DOI that resolves. The catch is that the DOI frequently resolves to a different paper. In one benchmark, 99 of 116 resolving DOIs in AI-written clinical answers pointed to the wrong article.

I wrote up a manual, five-minute way to check if a citation is real (full guide here). This post is about the developer side: which parts of that method can be automated, and how far you can get with public scholarly APIs and no server of your own.

The manual algorithm

  1. Search the exact title.
  2. Resolve the identifier and compare fields (title, first author, year, journal).
  3. Verify the journal issue and pages.
  4. Verify the author's publication record.
  5. Confirm the source supports the claim.

Steps 1–4 are lookups and comparisons, so they can be automated. Step 5 means reading the source and judging whether it supports the claim, which is a different problem.

Try steps 1 and 2 yourself

Most of this runs on free, public, CORS-friendly APIs. Resolving a DOI to its metadata with Crossref:

const doi = "10.1038/nature14539";
const res = await fetch(`https://api.crossref.org/works/${encodeURIComponent(doi)}`);
const { message } = await res.json();
console.log(message.title?.[0], message.author?.[0]?.family, message.issued?.["date-parts"]?.[0]?.[0]);

Searching OpenAlex by title:

const title = "Deep learning";
const url = `https://api.openalex.org/works?filter=title.search:${encodeURIComponent(title)}&per-page=5`;
const { results } = await (await fetch(url)).json();
results.forEach(w => console.log(w.display_name, w.publication_year, w.doi));

The important part isn't the fetch. It's the comparison. A resolving DOI tells you nothing until you diff the returned title, authors and year against what the citation claims.

How Fake Citation Detector structures it

Fake Citation Detector is the free Chrome/Edge extension I built around this idea. At a high level:

  • Identifier first. If a reference has a DOI, ISBN, arXiv ID or PMID, that's looked up first.
  • Then title + authors across Crossref, OpenAlex, DataCite, PubMed, Open Library and Semantic Scholar, because no single index covers everything (books, datasets and preprints live in different places).
  • Four outcome states instead of a boolean: confirmed, wrong details, not found, web source. A binary real/fake label would be dishonest, since "not found" often just means "not indexed".
  • Field-level diffs. Wrong details are rendered side by side (cited year vs. real year, and so on), with a one-click corrected reference in seven citation styles.
  • Web sources are fetched and compared, and dead links are looked up in the Internet Archive.
  • No backend. The browser talks to the public databases directly. No accounts, no proxy, and no reference list sitting on someone else's server.

Evaluation

In an October 2026 test on 397 references in seven citation styles:

  • 0 of 222 real references marked "not confirmed" (no false alarms)
  • 120 of 120 AI-invented references caught
  • 53 of 55 real references with one deliberately altered detail caught

Known limits

It doesn't do step 5, and "not confirmed" is a prompt to investigate, never proof of fabrication.

If you're building anything that generates or ingests references (RAG pipelines, writing tools, docs generators), the manual method is worth reading: it doubles as a spec for what your validator should check.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.