Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 3 min read

I shipped 44 tools. The audit said merge 18 of them. Here's the first merge.

Over a few weeks I shipped 44 small open-source tools. Then I did the thing you do when the shelf gets crowded: I audited my own fleet. The audit's verdict was uncomfortable and correct β€” 18 of the 43 tools were quiet, n

Over a few weeks I shipped 44 small open-source tools. Then I did the thing you do when the shelf gets crowded: I audited my own fleet. The audit's verdict was uncomfortable and correct β€” 18 of the 43 tools were quiet, not because they were bad, but because each one answered a fifth of a question. Nobody installs five separate CLIs to ask "is this AI-agent setup safe to open?"

So the audit ordered a merge: 18 quiet repos into 4 flagships. preflight-agent is the first one to ship. (v1.0.2, a documentation refresh, is out today.)

What got merged

Five published auditors, each answering one preflight question about an AI-agent setup:

  • clone-guard β€” what executes when this folder is opened, installed, or committed to? (Claude/VS Code/devcontainer configs, npm install scripts, direnv, git and husky hooks β€” every command shown verbatim)
  • secret-guard β€” which agent/MCP config files hold inline secrets? (Findings by key path and fingerprint; the value is never printed. Plus the one writing command in the suite: fix, which rewrites literals as ${ENV_VAR} references β€” dry run by default, timestamped backup, re-scanned before reporting.)
  • hook-doctor β€” which Claude Code hooks can never fire, can never block, or point at files that don't exist?
  • fleet-audit β€” in a folder of agent definitions: description collisions, duplicate names, tool grants that contradict the promised job.
  • skill-sprawl β€” in a tree of skills: cross-scope duplicate names, discovery collisions, oversized bodies, a discovery-budget forecast.

Now:

pipx install preflight-agent
preflight-agent scan ~/my-project

One command runs all five and prints one combined receipt. A module with nothing to audit says so β€” skipped, with its reason β€” instead of reporting clean. Exit code: 0 clean, 1 findings, 2 error, so it gates a script or a habit, not just a read.

The merge changed no engine

This is the part I'd want scrutinised, so it's stated plainly: each module's engine is byte-identical to its standalone tool's. No check, verdict, threshold or honesty limit moved. Each module keeps its own CLI shape behind the suite command (preflight-agent secret-guard scan FILE...), its own verdict vocabulary, and even its own version number (v0.1.0) inside its reports β€” the suite is v1.0.2, the engines are what they were.

What the merge actually added:

  • a dispatcher that routes to the five engines,
  • scan β€” target discovery per module (settings, configs, agents, skills, wherever a normal setup keeps them) and the combined receipt,
  • 24 new tests. The five source suites carried over 276 tests untouched, so the suite ships at 300 tests, all green.

The demo fixture plants exactly one problem per module β€” an .envrc that runs on cd, a literal key in .mcp.json, a guard hook pointing at a script that doesn't exist, an agent name defined twice, a skill name defined in two scopes β€” and the README shows the real receipt verbatim. Build the fixture, run the scan, diff it yourself.

Watch it run (29 seconds): the demo video runs that exact fixture scan β€” all five modules report their planted problem in one combined receipt, closing on the verdict. Watch the demo

The disclosed exception

"Read-only" has one footnote, and it's printed in the README and the changelog rather than buried: secret-guard fix --apply writes. That's its job β€” it rewrites the config files you name, replacing inline secrets with environment-variable references. Dry run unless --apply, a timestamped backup beside each changed file, and the rewrite is re-scanned before it's reported. The combined scan never calls it.

Honest limits

Static evidence only. clone-guard reports surfaces that would execute, not what code does once running. secret-guard parses JSON configs; TOML/YAML and ordinary key names outside env blocks are out of scope, and a secret that was ever committed stays leaked β€” rotate it. hook-doctor never runs a hook, so a script that exists but misbehaves is invisible. fleet-audit and skill-sprawl similarity is a token-set Jaccard heuristic at stated thresholds β€” a reported figure, not a platform guarantee. A CLEAN verdict means "none of the checked problems were found", never "this setup is safe".

It's MIT, zero dependencies, stdlib only, no network, and it prints no score β€” a receipt, not a rating.

Repo: https://github.com/jasveerai88/preflight-agent

Three more merges are queued behind this one, on the same audit's orders. Feedback I'd most value: setups the scan misjudges β€” a surface a module misses, a verdict the evidence doesn't support. That feedback is what the fixtures are for.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.