A log where every row says allowed proves nothing

An auditor asked one team to show a refusal. The team produced a log with 1,509 rows. Every row said allowed. The auditor's question was not "is this signed" — it was "what did you ever stop".

That story is now a small industry cliché, and teams have adapted: they add denials. Which is where the interesting failure starts. A log full of denials can be just as empty as a log with none, and it survives review much longer, because nobody looks at a denial twice.

Here is the version of that problem I hit in my own code.

The row looks complete

A mandate checker decides whether an agent may spend. Each decision is appended to a ledger. A row from ours:

{
  "at": 1761740400,
  "intent": "pay_invoice",
  "verdict": "DENY",
  "rule": "amount",
  "payee_id": "0xAAA",
  "audience": "client-7",
  "amount_minor": 1250,
  "currency": "USDC"
}

Nine fields, a named rule, the amount, the payee, the audience. It answers "what happened" and "which rule fired". And it cannot answer the only question a reviewer actually has:

1250 out of what?

The threshold the amount was compared against is mandate.max_amount_minor. It is not in the row. Neither is the mandate version, nor the value the rule read. So the row says "this was over the limit" and leaves the limit to memory.

Two consequences, both practical.

First, the row is not falsifiable later. If the mandate was 1000 that day and 5000 a month later, the row still reads the same. Anyone reviewing the log after the mandate file changed has no way to recompute the decision. They can verify the signature. They cannot verify the judgement.

Second, and worse, a row like this cannot distinguish a real denial from a broken check. Which brings me to the second failure.

Denials that are technically true

Under a post about audit logs, someone described their own log: 119,636 rows, 37 denials. All signed. 36 of the 37 had the same reason — output empty or not a string — while the tool's payload actually carried the text under result.text, not output. The checker looked in the wrong place, found nothing, and denied.

Every one of those 36 rows is cryptographically valid. The signature is correct, the chain is intact, the reason field is populated, the policy id is named. And all 36 are noise: they record a bug in the checker, not a decision about the agent.

That is the trap. A signature proves what the checker read. It does not prove what the subject did. If the checker reads the wrong field, you get a beautifully attested log of the wrong question.

It also cuts the other way: 36 false denials hide the one real one perfectly. A reviewer who skims reasons sees the same string 36 times and stops reading.

The minimal fix

Record what was compared, not only what was decided. Three fields, inside the signed body, on the same chain:

{
  "verdict": "DENY",
  "rule": "amount",
  "rule_input": {"amount_minor": 1250, "max_amount_minor": 1000},
  "mandate_hash": "sha256:1f4c…",
  "policy_version": 3
}

• rule_input — the two values the comparison actually used. Not the path, the values.
• mandate_hash — which mandate this decision was made against. The mandate itself can live elsewhere; the row only needs to point at the exact version.
• policy_version — so that when the rule order changes, old rows are not silently reinterpreted.

Then the check that makes the log worth reading is one line per row:

for row in ledger:
    if row["verdict"] == "DENY":
        assert row.get("rule_input"), f"unfalsifiable denial: {row['id']}"

A denial with no inputs is a refusal to explain. It should fail the same way a bad signature fails.

For the "wrong field" problem there is no schema fix — rule_input at least makes it visible, because the row will show {} or null where a value was expected, and a human reading twenty of those in a row will notice the pattern that a machine happily signs.

What this still does not fix

Being honest about the ceiling matters more than the fix.

• Omission. A complete, valid chain can be a subset of what happened. Chaining protects against editing, not against silence. If the process that writes the log can choose not to write, no hash helps.
• Selection. Nothing stops a system from showing a different chain to a different reviewer. Publishing a chain head somewhere you do not control is the only partial answer, and it costs money and explanation.
• Truth of the policy. The log proves the rule was applied as written. It does not prove the rule was right. A perfectly enforced bad limit is a perfectly enforceable bad limit.

The gap in our own code

The version of the record I quoted at the top is the one shipping today. rule_input and mandate_hash are proposed, not shipped — I am writing this before the commit exists, because the finding is more useful than the announcement. When the field lands, the article gets a line with the commit and the log stays honest.

If you are auditing an agent's spend log right now, the test is one question: pick any denial and ask what value it compared against. If the answer requires opening a file that has since changed, the denial is decoration.

Signed refusals with the values they compared: attest.rta.lat/deny-evidence — the record there carries meta.requested_amount and meta.allowed_max for exactly this reason.