I Open-Sourced a Tool That Proves What Your AI Agent Actually Did

AI agents are getting scary capable. Ask one to run a pentest, chain some automated operations, whatever — it's fast, it's thorough, and when it's done, it hands you a report.

Why should you believe that report?

I treated that question as a threat-modeling problem, not a rhetorical one. Three failure paths, one by one:

1. Logs fail. The log runs in the same process and under the same privileges as the agent. Anything the agent can write, it can rewrite. A process that can edit its own log files produces evidence with zero forensic value for the question "what did this process do?" The basic requirement of evidence is independence from the party being examined. Same-process logging violates it completely.

2. Self-reports fail. A report is a self-description, and self-descriptions can diverge from facts — hallucination, context loss, whatever. LLM output is sampled: ask twice about the same operation and you may get two different reports. Using a nondeterministic generation as evidence of deterministic behavior is a category error.

3. After-the-fact logging fails. Backfilling a "compliance" record afterwards is technically trivial. Timestamps come from a local clock — it says whatever time you want it to say. Without an independent time anchor, you can't defend against "did it later, logged it earlier" or "never did it, logged it anyway."

Put together: to prove what an agent did, the evidence must be independent of the thing being proven. Not from its own process, not its own words, not on a timeline it controls.

I built this trust layer for my own autonomous pentest agent system, and today it's open source: cyberseccat-fence (GitHub: TLotU-lab/cyberseccat-fence, Apache 2.0).

Every number in this article carries a bracket tag: [git] means there's a commit you can check, [tested] means I personally re-ran it, [fossil] means the file timestamp is verifiable. That's the discipline I use across this project, and this article follows the same rules — every claim maps to an artifact in the repo.

What's in the box

fence is the trust layer of CyberSecCat, my autonomous pentest agent system. Four things are public:

1. An independent verifier (fence/verify_bundle.py, ~600 lines): verifies every structural and cryptographic assertion of an evidence bundle (we call it an Audit Pack) locally. Zero network, zero accounts, and the party that produced the bundle doesn't have to cooperate.
2. The authorization contract data model (scope_engine/token_schema.py): what a valid authorization looks like — tokens, asset scope, action types, time windows, approvers, a state machine. This module is the first line of code in the entire project [git].
3. The scope enforcement test set (testset/): 87 verdict cases (v1.0) plus 25 drift-induction cases (v1.1), with expected verdicts preregistered and frozen before running [git: tests/audit/testset_prereg].
4. The cross-validation record (tests/audit/): 91 vectors cross-checked against an external reference implementation, three re-run rounds, vector-by-vector identical [git: interop_asqav_91_*].

What the agent did, whether it stayed inside its authorization — don't take my word for it. Run the verifier yourself. (The repo README is bilingual, Chinese and English.)

How to use it

Two third-party dependencies (pydantic, cryptography — both permissive licenses), Python ≥3.11 (tested on 3.12 and 3.14 [tested]):

pip install pydantic cryptography

# Step 1: generate a synthetic demo bundle locally (one command, a few seconds, no network)
python samples/generate_demo_pack.py

# Step 2, mode two (default): structure + hash chains + Ed25519 signatures
python -m fence.verify_bundle samples/audit-pack.json --mode two

# Step 2, mode one: no signature checks — structure, chains, and anchors only
# (the zero-dependency form, for air-gapped machines)
python -m fence.verify_bundle samples/audit-pack.json --mode one

The demo bundle is a locally-generated synthetic sample, not real pentest evidence. Expected results for a fresh sample [tested]: mode two → WARN (13 checks passed, 0 failed, 0 deferred; the only WARN is that the timestamp anchor hasn't confirmed on-chain yet); mode one → PARTIAL (4 signature checks deferred).

How to verify it

Three levels, increasing effort.

Level 1 — run the sample. The commands above. Two minutes, zero network. You'll get an axis-by-axis report, not a single pass/fail boolean.

Level 2 — audit the claims. The README has a claims table: every assertion ("out-of-scope actions 100% blocked, in-scope zero false positives", "verification doesn't depend on the producing party", "cross-validation stable across three rounds") maps to an inspectable artifact in the repo [git: tests/audit/]. The mapping from claims to evidence is itself auditable.

Level 3 — do the cross-validation yourself. The 91-vector corpus lives in the reference implementation's repo (the corpus stays on their side — that's what makes it a cross-check), with reproduction steps, SHA-256 of the corpus, and file-tree digests in the README. 90 vectors match strictly, 1 tolerable divergence with root-cause analysis in the repo, three re-run rounds vector-by-vector identical [git]. 90 strict + 1 tolerable — counted to the last digit, not rounded.

What's NOT open — and why

The verdict logic (prefix trees, gateways, parsers), the L1–L3 physical-layer enforcement, the agent runtime, the tool layer: not in the repo, and no open-sourcing promise. Publishing enforcement logic is publishing a bypass reference. The authorization contract model is public because a reference implementation needs an interface to build against. That's a deliberate boundary, not a "coming soon" list.

The README also carries a list of unfinished items — cryptographic re-verification of anchored timestamps, a bundle-level verdict discriminator, receipt chain sequence membership, and a deferred-dependency guard — each with its current status. What state it's in is what the README says it's in.

What this is: the opening piece of a series. CyberSecCat went from a weekend MVP to where it is now in three and a half months; fence is the public artifact of the "how do I control it, how do I prove it" half of that journey. Future posts will cover how this thing grew — starting with one weekend: how I got an LLM to help me build the first security agent, and why the first generation's retirement list includes an authorization check that was written as a TODO and never fired.

Project: GitHub TLotU-lab/cyberseccat-fence — bilingual README, claims table, reproduction steps, and the unfinished-items list are all in the repo. Bring your skepticism and run it.