My security tool aced its own exam. Then I ran it on real repos

TL;DR - I'm building croak 🐸, a tool that checks what a repo or a package install would do before any of its code runs. On my own eval it caught every attack with zero false alarms. On its first run against 13 popular open-source repos, it flagged zod and esbuild as malicious. This post is about why a self-written eval wasn't enough, and the two changes that real code forced.

The problem croak is for

npm install doesn't just download code. It runs it: postinstall scripts, native builds, whatever a package decides to do. Opening a folder in VS Code can run a task too. And AI coding agents now do both all day, usually without reading anything first.

So I'm building a gate that inspects a repo as plain data and gives a verdict before anything executes:

• 🟢 allow: nothing suspicious found (not a guarantee)
• 🟡 warn: behavior that needs your attention (never "safe")
• 🔴 block: strong evidence of an attack

It's early-stage. Seven detectors so far: install scripts, VS Code autorun tasks, hidden code, obfuscation, dangerous git config, committed node_modules, and files too large to scan.

Eval first

I wrote the eval before the first detector. Every case is a small fixture repo with an expected verdict:

{
  "id": "d-postinstall-curl-pipe",
  "threat": "D",           // fake "test assignment" repos
  "expect": { "verdict": "block", "rules": ["repo/lifecycle-script"] }
}

Payloads are inert: they point at reserved .invalid domains or just echo. Two rules turned out to matter a lot.

1. Every attack gets a benign twin. A postinstall that pipes curl into sh sits next to one that runs prisma generate. A 413-character line hiding eval after 320 spaces sits next to an honest 2,875-character Tailwind safelist. Without twins, a detector that just flags "has a postinstall" or "has a long line" looks perfect.

2. You can't let the eval be gamed. My review agent wrote a detector that matched only the SYNTHETIC marker comment in my fixtures. It scored 4/4 with zero false alarms, because only the malicious fixtures had the marker. Now every fixture, benign or malicious, carries it (a test enforces that), and a lint rule forbids detector code from mentioning it.

After a few iterations the eval looked great:

croak eval — 32 cases (16 malicious, 16 benign)
  flag recall   16/16  100%
  false alarms   0/16    0%

I wrote every one of those 32 cases myself. That's the catch.

The first real run

I pinned 13 popular repositories by commit (axios, express, zod, hono, ky, husky, chalk, create-t3-app, trpc, esbuild, sharp, vscode-extension-samples, vue core) and scanned them. 7,455 files took about one second.

zod and esbuild are not malware. Two false blocks on the first real run.

Lesson 1: "near" means near

Both blocks came from the obfuscation detector:

[block] zod/packages/zod/src/v3/types.ts:607  repo/obfuscation
        obfuscated code: escapes (line 607), next to eval or decoding

Line 607 of zod is a Unicode regex: dense \u… escapes, a weak signal I deliberately don't treat as obfuscation on its own. But the file also calls atob() for base64 validation, thousands of lines away. My rule said "weak signal + eval/decoding in the file → block". The file was the wrong unit.

The fix: a weak signal only escalates when eval or decoding is within three lines of it. const font = "…base64…"; eval(atob(font)) on adjacent lines still blocks. A regex and an unrelated atob() in different corners of a big file don't. Strong signals (_0x identifiers, string-array rotation, calls through computed globals) keep the file-wide rule.

My synthetic fixtures were all small files. That bug couldn't exist in my eval.

Lesson 2: a warning everyone ignores is worse than none

The second finding wasn't a bug. Seven of the 13 repos got 🟡 because their install scripts ran things like:

prepare: husky
prepare: npm run build      → tsc
postinstall: prisma generate

Technically true: code runs on install. In practice it's noise, and noise trains people to skip the yellow line, which defeats the tool.

The new rule is strict on purpose. A script is only an info note when every command is a known tool, or an npm run call to a script that is itself routine, and there is nothing else: no network, no local files, no inline code, no suspicious code signals. Anything unknown stays 🟡. So cd client && npm install (a nested install runs other packages' scripts) is still a warning, and prisma generate && curl … | bash is still a block. Recognizing a known tool never launders the rest of the command.

Before and after

The 26 warnings that remain are all explainable: nested installs, an unknown tool that downloads files at install time, install scripts that run local code. That's exactly what 🟡 is for.

The synthetic eval didn't move: still 100% flagged, zero false alarms. That's the point. Both kinds of runs have to pass at once.

What I'd tell myself a week ago

• A synthetic eval proves your detectors match your imagination. It's necessary, but on its own it isn't enough. Pin a few real repos and run them on every significant change.
• Write the benign twin first. If you can't describe what honest code looks like, you can't tell your detector what not to flag.
• Assume your eval can be gamed, and check whether it can, ideally with a reviewer that has no stake in the score.
• Treat noise as a defect, not a cost of being careful.

What croak doesn't do (yet)

It doesn't follow require chains past the file a script runs directly. It doesn't check package installs yet (that's next), it isn't an antivirus, and there's no AI layer yet. When one comes, it will only be allowed to raise a verdict, never lower one. All of that is written down in the repo, because a security tool that oversells itself is its own kind of vulnerability.

croak is TypeScript with zero runtime dependencies, and early-stage. The repo is going public soon: [link]. I'll write about the install check and the cryptography chapter (it starts with my 2013 thesis) in later posts.

Thanks for reading. If you've built evals for detectors or LLM features, I'd love to hear how you keep them honest. 🐸