I scanned 35 real SKILL.md files — 16 tripped a rule, 10 of them Anthropic's official skills

Your AI agent reads files written by strangers, and then does what they say. Every SKILL.md it loads is a set of natural-language instructions your agent will follow — and almost nobody runs them past a checker first.

So we did. Here's the whole methodology, reproducible in under a minute.

The experiment

We took 35 real, public SKILL.md files:

• the 20 skills in Anthropic's official skills repository
• 15 from superpowers, one of the most popular community skill collections on GitHub

Then we ran them through SkillSentry's static rule layer only:

npm install -g @ulkoro/skillsentry
skillsentry . --json --no-ai

No model review, no judgement calls — just the fourteen pattern checks documented in the Rulebook. Total scan time for all 35 files: 161 milliseconds.

One disclaimer: we build SkillSentry, so treat this as a chef tasting their own soup in public. Every number below comes from the JSON output of that one command, and both repos are public — run it yourself and they'll reproduce.

The numbers

36 findings total: 1 critical, 16 high, 7 medium, 12 low. The median file scored a risk of zero — the risk concentrates in specific, nameable patterns:

• SS-002 (outbound URLs next to data-movement language): 11 hits
• SS-006 (cross-tool routing hijack), SS-007 (personal data collection), SS-011 (package installs from non-official sources): 6 each
• SS-005 (hidden instruction channels): 2, one of them the single critical

Read that again: the official ones

Ten of the sixteen flagged skills are Anthropic's official ones — including all four dangerous verdicts.

The worst scorer (risk 79) was an official skill whose job is fetching a published catalog over HTTPS: ten SS-002 hits for doing exactly what it says on the tin. The docx and pptx skills tripped SS-011 for the pattern "try require('docx'), and only if that fails run npm install docx" — a convenience today, and the exact door a typosquat walks through the day a dependency goes missing.

To be unambiguous: none of this is an attack. These are well-built skills by a team that ships threat models most projects never write down. The scanner fired anyway because it doesn't score intent — it scores shape. And the shape of "fetch this remote resource," "install this package if it's missing," and "prefer my tool over that one" is precisely the shape every documented skill attack has used.

The critical, on the skill that creates skills

The single critical finding landed on skill-creator — the official skill whose job is writing new skills. SS-005 flags content that smuggles instructions through channels a skimming human reads as boilerplate.

That's not an indictment; it's the trade in one anecdote. A pattern scanner must fire on patterns even in the most trustworthy file on the list — because the day a malicious skill copies that exact shape (and someone will), the detector must already be deaf to reputation. False positives on the good guys are the price of catching the bad ones early. A smoke detector that stays silent in the kitchen isn't discriminating; it's broken.

So is SKILL.md broken?

No. SKILL.md is where npm was before lockfiles and audit: an ecosystem shipping a new supply chain faster than the hygiene layer that eventually made dependency management survivable. The format is fine. What's missing is the habit of running the text past a checker before an agent runs the text. Nobody audits a README they'd never read — and skills are READMEs that execute.

Run it on your own shelf

npm install -g @ulkoro/skillsentry
skillsentry .          # scan a repo
skillsentry diff old new   # what changed between snapshots?

Or paste a single SKILL.md into the web scanner. Clean takes five seconds and changes nothing. A hit gives you the exact rule, the evidence line, and how to fix it — before your agent follows it.

The full write-up with the complete methodology is on intothe.tech (canonical source). The fourteen rules, with evidence and remediations, live in the Rulebook.