Your AI agent is reading the wrong 25 million tokens

I built a small CLI, dev-diagnosis, that answers one question about a repo:
what is my coding agent reading that it shouldn't be — and what does that
cost in tokens? Then I pointed it at 10 of the most-starred JavaScript and
TypeScript repos on GitHub. The results are below. Every number comes from a
real run of the published tool (v0.2.2), not from my head.

How it works

npx dev-diagnosis-cli all

One command, no install, no config. It walks the repo, flags the files an
agent should not read — build output, lockfiles, logs, oversized files — and
prints a 0–100 score, the noisy files, and the token cost ("Estimated tokens
wasted on this noise", noise bytes ÷ 4, the tool's own label). dev-diagnosis
all --fix writes the suggested .aiignore entries and re-scores.

The results

Star counts were fetched from the GitHub API on 2026-11-13 (the React repo now lives at react/react).

Across all ten repos that's ~45.7M tokens wasted per full read. Median
score 95.5. And four of the ten (Node, Next.js, Deno, TypeScript) hit the
tool's 10,000-file scan cap, so their numbers are lower bounds — the real
waste is higher.

Three things I didn't expect

1. The biggest waste is stuff nobody wrote. Node's top item is an 11 MB
ICU data file. Deno's is a 10 MB JSON test fixture. TypeScript's is a 4 MB Go
test file. These aren't source code anyone reviews — they're vendored data and
test fixtures. But a naive "read the repo" prompt feeds all of it to the
agent, every turn, at full price.

2. React's problem is 47 copies of the same file. Nearly every fixture
directory carries its own yarn.lock — 6.7 MB, 16.5% of the repo's bytes.
The fix is three lines in .aiignore. I ran it on a copy of the repo to check:

The waste is structural (monorepo fixtures), the fix is trivial. That's the
whole point of the tool: it finds the three lines worth writing.

3. Small, disciplined repos don't have this problem at all. Express scored
100 with zero noise across 216 files. Svelte's entire waste is one lockfile.
This is a scale problem: the bigger and more monorepo-shaped the project, the
more an agent pays for files it will never use.

What the tool can't do (honest limitations)

• The token numbers are estimates, not bills. Noise bytes ÷ 4 is a rough rule of thumb. Real tokenizers differ, minified files tokenize differently, and your agent may not read the whole repo anyway. Treat the numbers as comparable across repos, not as exact costs.
• The scan stops at 10,000 files. Node, Next.js, Deno and TypeScript all hit the cap, so their scores and token counts cover only the first 10,000 files. Their real waste is higher than the table says.
• "Noise" is the tool's judgement. A lockfile is noise when an agent is writing code; a bundled .d.ts might be exactly what a human wants to inspect. The tool flags by size and category — you decide what to exclude. Nothing is deleted or changed unless you run --fix.
• It only measures the repo on disk. It doesn't know what your agent actually read, and it can't diagnose a remote CI run without logs. The path, cache and flakes checks cover commands, build caches and flaky tests, but they need their inputs passed in.

Try it

npx dev-diagnosis-cli all

GitHub: https://github.com/WJDdidi-1/dev-doctor —
npm: https://www.npmjs.com/package/dev-diagnosis-cli

If you run it on your own repo, I'd genuinely like to know what it finds —
especially if the numbers look wrong. That's the fastest way to make the tool
better.