Dev.to AI 🤖 Ai 👁 0 📖 3 min read

Slopsquatting, measured: 3.83% of 2,532 AI package suggestions didn't exist

The headline number going around this week: AI coding models hallucinate package names about 20% of the time. It comes from a Virginia Tech / Oklahoma / Texas study that ran 16 models over 576,000 code samples and found

The headline number going around this week: AI coding models hallucinate package names about 20% of the time. It comes from a Virginia Tech / Oklahoma / Texas study that ran 16 models over 576,000 code samples and found ~205,000 unique hallucinated package names — with 43% of the same fake names repeating across identical prompts. That repeatability is what makes slopsquatting (registering the fake name before a victim installs it) actually viable.

Big round numbers make me twitch. So I measured it myself.

Method

In October I ran a panel: 2,532 package suggestions drawn across four model families, each checked against the real registries (npm, PyPI, crates.io) to see whether the name exists. That's the whole method — no exotic prompt engineering, just "the model suggested it, does it exist."

Result: 3.83% of suggestions didn't exist.

Update: replicated in 2026

A 2026 replication across five frontier models measured hallucination rates of 4.62%–6.10% — lower than the 2025 study's ~19.7%, but not retired. It found 127 package names all five models invent identically. And the panel split behind my 3.83%: 0.17% on routine prompts, 5.83% on fast-moving ones.

The honest comparison

My 3.83% is not a refutation of their ~20%. Different models, different prompts, different month. Treat mine as one more data point, not a takedown. What both numbers agree on: this is not rare, and the dangerous part isn't the hallucination rate — it's the repeat rate. A fake name suggested once is noise. A fake name suggested every time, for everyone, is a target list.

What actually helps

Verification at install time. Not a vibe check, a lookup: before pip install / npm install runs on anything an AI suggested, confirm the name exists in the registry.

I built the boring version of that: pkgguard, an open-source pre-install gate that blocks nonexistent package names before they install (repo: rxslice/pkgguard-API on GitHub). October panel data: 1.0% false-block rate, 7/7 recall on known hallucination test cases. If you don't want to run it yourself, there's a free managed HALLUX API on blvkware.dev that does the same check.

The fix for this class of attack is refreshingly unglamorous: check the name exists before you install it.

Since the study: it's live now

The numbers above were measured against a study. This week the story moved from research to live attacks:

  • GhostAction (reported Oct 15, via GitGuardian): a wave running since late August spiked Oct 8–9, compromising ~1,900 GitHub accounts and targeting 30,000+ repos. The payload steals CI/CD secrets, cloud credentials, registry tokens — and AI provider API keys.
  • StepSecurity's 12-month count (published Oct 16): 56 confirmed open-source supply-chain attacks in a year. The rate went from ~1/month to ~9/month — roughly one every three days since March.
  • Tensorlake's npm package (Oct 8): compromised in the ChainDrop/Shai-Hulud campaign. The stealer harvested npm/GitHub/AWS/K8s/SSH/crypto creds — and config files for AI tools like Anthropic Claude. Agent infrastructure is in the blast radius now.

This week: it stopped being about packages

Two incidents disclosed Oct 16 (CSO Online, via AIR Security's research lab):

  • An official OpenAI skill shipped a hallucinated domain. The Render deployment skill — in OpenAI's own vetted skills repo — told production apps to send their tokens to render-internal.com, a domain nobody owned. The researchers registered it, pointed it at a listening sinkhole, and watched real production tokens and API calls roll in. The flaw sat there since January. Slopsquatting was never just package names; it's any identifier an agent trusts without checking.
  • A trojanized skill on skills.sh hit 1.7M installs. It typosquatted Paperclip and Browser Use, then instructed agents to install a credential stealer from GitHub. The agent becomes the attacker's installer: the poisoned instruction rides in as trusted context, and the agent does the clicking.

Same pattern in every case: an agent trusted a name it never verified. Check the name before you install it — and check what your agent is reading, too.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.