4 secrets in 12 beat both scanners — I planted them

I planted 12 fake secrets in Claude Code settings files and ran the exact scanner versions Apiiro used. Gitleaks 8.30.0 caught 6 of them, TruffleHog 3.91.0 caught 7, and 4 of the 12 — every one of them a bare password with no provider format — beat both tools. The controls, a placeholder password and a file with no secrets at all, produced zero false positives.

What Apiiro found

Apiiro's November 4 post reports 104,185 public GitHub copies of .claude/settings.local.json across 103,617 repositories. Of those, 1,122 files (1.08%, they round to 1.1%) hold at least one real secret: 1,379 distinct secrets, split into 636 provider keys and 743 passwords or tokens for public hosts. The mechanism is brutally simple: when you approve a command with "Don't ask again," Claude Code saves the exact command string as an allow rule in that file. The rule is the command string — so a password typed once into a psql or sshpass invocation is stored verbatim, and then the file gets committed.

Two of Apiiro's numbers deserve a second look before I build on them. First, their per-type chart sums to 1,384, not the headline 1,379 — five secrets over, a small inconsistency in their own display. Second, their scanner rates check out against their own totals: Gitleaks flagged 1,026 of 1,379 (74.4%), TruffleHog 987 (71.6%), either tool 1,164 (84.4%), and neither tool flagged anything in 215 files (19.2%). The arithmetic is clean; the 1,384-vs-1,379 gap is the only crack I found.

I also spot-checked the scale independently. Sourcegraph's public index — which skews toward starred repos, so this is a floor, not a recount — holds 928 copies of the file across 841 repositories. GitHub-wide code search is what gets you to Apiiro's 104,185. Either way, the file is all over public GitHub.

The experiment: 12 files, 12 fake secrets

I built 12 realistic .claude/settings.local.json files, one secret each, mirroring Apiiro's categories. Every value is random and fake, generated locally: example.com hosts, TEST-NET 203.0.113.x addresses, no real credentials anywhere. Each file also carries two benign allow rules (npm run build, git status) so the scanners see a normal file, not a bare secret. Two controls: one with PGPASSWORD="password" against localhost, one with no secret at all.

01  PGPASSWORD db password in a psql migration command
02  sshpass SSH password (ssh [email protected])
03  Bearer JWT in a curl Authorization header
04  AWS access key + secret key as env vars in a command
05  GitHub personal access token (ghp_) in a curl header
06  Supabase sbp_ token + service-role JWT in mcpServers env
07  Google API key (AIza...) in a Maps URL
08  Anthropic API key (sk-ant-api03-...) in a CLI invocation
09  MySQL password via mysql -p'...'
10  npm auth token in an npm config command
11  Database password inside a postgres:// URL
12  Stripe live secret key (sk_live_...) in a curl -u

Then I ran Gitleaks 8.30.0 and TruffleHog 3.91.0 — the exact versions Apiiro used — with default configs and no verification, over the directory.

The scoreboard

file  secret type              gitleaks 8.30.0     trufflehog 3.91.0
01    PGPASSWORD password      MISS                MISS
02    sshpass password         MISS                MISS
03    Bearer JWT               HIT (jwt)           MISS
04    AWS AKIA key             HIT (aws + generic) HIT (AWS)
05    GitHub ghp_ token        HIT (github-pat)    HIT (Github)
06    Supabase sbp_ + JWT      HIT (generic x2)    HIT (SupabaseToken)
07    Google AIza key          MISS                MISS
08    Anthropic sk-ant key     HIT                 HIT (Anthropic)
09    mysql -p password        MISS                MISS
10    npm token                MISS                HIT (NpmToken)
11    postgres:// password     MISS                HIT (Postgres)
12    Stripe sk_live_ key      HIT (stripe)        HIT (Stripe)

Gitleaks:   6 of 12 files flagged (8 findings)
TruffleHog: 7 of 12 files flagged (7 findings)
Either:     8 of 12 (66.7%)   Both missed: 4 of 12 (33.3%)
Controls:   0 false positives from either tool

The four survivors — PGPASSWORD, sshpass, mysql -p, and the Google key — are the point. Three are passwords with no provider format: no AKIA prefix, no ghp_ prefix, no fixed length, nothing for a regex to grab onto. The fourth, the Google key, has a format (AIza + 35 chars) but neither tool's default config has a rule that fires on it in this context. Apiiro's 215 unflaggable files are the same shape: the secret sits inside a command string, not in a KEY=value assignment a scanner recognizes.

Two misses that taught me something

Not every miss is the scanner's fault. My first run had four unrealistic fakes, and the scanners were right to ignore them:

• My AWS key was invalid. I generated AKIA + 16 random uppercase/digits. Gitleaks' AWS rule only accepts [A-Z2-7]{16} — base32. That is not a scanner quirk: real AWS access key IDs are base32, so they never contain 0, 1, 8, or 9. My key did. I regenerated with the right alphabet and it flagged immediately.
• My JWT was invalid too. Gitleaks' JWT rule requires the payload segment to start with ey — base64 for {". My first payload was random bytes. Real JWT payloads are JSON, so they always start with eyJ. Fixed, flagged.
• Anthropic keys end in AA. Gitleaks' rule requires sk-ant-api03- + 93 chars + AA. My first key lacked the suffix. Fixed, flagged.
• The npm miss is real, and it's subtle. The npm token is detectable — in plain text. Gitleaks' npm rule requires the character after the 36-char token to be a quote, whitespace, semicolon, or end of line. Inside a Bash(...) allow rule, the token is followed by ) or by \" — neither qualifies. I verified it: the same token in a .txt file flags instantly; inside the settings JSON it doesn't. The Bash(...) wrapper itself breaks the scanner's boundary assumption. TruffleHog's npm detector has no such terminator requirement, so it caught it.

The honest summary: format-valid provider keys are caught reliably by at least one tool. Bare passwords in command strings are caught by neither. That matches Apiiro's 84% either-tool rate on their 1,379 — and it means the 16% they couldn't flag isn't a scanner tuning problem. It's structural.

The fix I couldn't verify

Apiiro says Anthropic replied the same day, planning a fix to prevent committing .claude/settings.local.json, shipping in Claude Code 2.0.54 and announced November 5. I checked the public record and it doesn't line up: npm dates 2.0.54 to November 25, 2025, the current release is 2.1.319 (November 6, 2026), and the release notes for 2.1.316 through 2.1.319 mention no settings.local.json or gitignore fix. Maybe the fix shipped under a different description, or in a version I haven't found — but as of today I cannot confirm it from the public release notes. Don't assume your install protects you.

What does protect you is boring and local. Claude Code adds the file to your global git excludes when it first writes it inside a repo — that rule lives on your machine, not in the project, so it doesn't survive a fresh clone, a new machine, or a folder that becomes a repo later. Check your own exposure with git log --all --oneline -- '**/settings.local.json' and git check-ignore -v .claude/settings.local.json (no output means git would happily commit it). Then: rotate anything that was ever in one of these files, purge it from history, add it to the repo's own .gitignore, and stop putting credentials in commands — pass them as environment variables so an approved command never contains the secret in the first place.

Sources: Apiiro, "Coding Agents Are Leaking Your Secrets to GitHub" (Nov 4, 2026) — 104,185 files, 1,122 with secrets, 1,379 secrets, scanner rates, disclosure timeline; Sourcegraph public code index (928 copies / 841 repos, queried Nov 7, 2026); npm registry metadata for @anthropic-ai/claude-code (2.0.54 published 2025-11-25, latest 2.1.319 on 2026-11-06); Claude Code GitHub releases 2.1.316–2.1.319. My experiment: 12 synthetic settings files + 2 controls, all values randomly generated fakes on example.com / TEST-NET hosts, scanned with Gitleaks 8.30.0 and TruffleHog 3.91.0 (default configs, no verification) on Nov 7, 2026. All counts above are my own runs.