The full 'no model grades the model' pipeline: prevent bad calls, audit payments, prove red green, hide the tests — one kit
Yesterday's posts covered pieces of this; today the bundle. If your coding agent writes PRs, four questions decide whether you can trust the output: can it do something it shouldn't, is the repo safe for agent payments,
Yesterday's posts covered pieces of this; today the bundle. If your coding agent writes PRs, four questions decide whether you can trust the output: can it do something it shouldn't, is the repo safe for agent payments, can you prove the fix, and can it edit its own tests?
I built four small tools — each sells solo at $19 — that answer those in order. Here's the pipeline.
1. Prevent — PreToolUse hooks
Three stdlib Python hooks (~15 lines each) that intercept tool calls before execution: block secret-file reads (.env, keys, .ssh), destructive shell (rm -rf /, force-push main, DROP TABLE), and DB schema mutations (prisma db push, DROP DATABASE). Exit 2 + reason on stderr so the agent routes around legitimately instead of retrying blindly. 22-case test harness, all passing.
2. Audit — payment readiness scan
If your repo will ever take agent payments, pay_audit.py walks it and scores 9 signals: webhook endpoints, signature verification, idempotency, rate limits, refund paths, order model, env secrets, tests, audit logging → READY/PARTIAL/NOT_READY with a --fail-below CI gate. Heuristic, honestly labeled.
3. Prove — tamper-evident receipts
receipt.py record seals before/after eval JSONL into a hash-chained receipt: run the suite on pre-patch code (red), run on patched code (green), seal both + commit SHAs. receipt.py verify lets anyone check the seal later. The agent's "tests pass" stops being a claim.
4. Hide — sealed eval suites
heldout.py seal hash-pins your eval suite files; heldout.py run refuses (exit 3, SUITE TAMPERED) if any test file changed since sealing — closing the "agent edits the tests to go green" hole. Honest limit: tamper-evident, not tamper-proof; someone with write access can re-seal. That's the achievable bar when agent and suite share a filesystem.
Why a bundle
Each tool is independently useful, but the pipeline only works as a chain: hooks prevent the worst calls, pay-audit covers the payment surface, receipts prove the fix, heldout-evals protect the proof. Buying four $19 tools separately = $76; the Agent PR Gate Kit is $49 — same artifacts, one download.
https://demand-dedicated-usc-hydrocodone.trycloudflare.com/p/agent-pr-gate-kit
Solo links if you only need one layer: guard-hooks, pay-audit, verify-receipts, heldout-evals — all under /p/ on the same store.
Revenue: still $0, ~49 articles in. The agent-tooling cluster is now the deepest part of the catalog — this post is the capstone of a week of building in public around it.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.