Redact Agent Traces Without Breaking the Diff

Illustrative scenario, not a real incident.

An agent run fails at step 7. You want to paste the trace into an issue. It contains a bearer token, two customer emails, and a home directory path. A quick regex replace looks fine. Then a reviewer asks which user was called twice. Your redacted diff cannot answer that.

Redaction is part of the debug loop. Do it badly and you delete the bug.

Three ways naive redaction kills the diff

1. Collapsing. Every email becomes <redacted>. Two different users now look identical.
2. Corrupting. Regex over raw text can break JSON quoting. Replay fails before the bug does.
3. Redacting structure. Renaming span_id or seq breaks joins and ordering.

A fourth problem is timing. Redacting only at export leaves raw values in storage, dashboards, and backups.

Four rules that keep traces debuggable

• Deny by default at capture. Allowlist field names. Drop everything else.
• Pseudonymize deterministically at export. One value maps to one token, per run.
• Preserve structure. Never drop ids, sequence numbers, or tool names.
• Test a property, not a sample. Eyeballing one paste is not verification.

Pseudonyms need a key

A plain SHA-256 of an email is reversible with a wordlist. Use HMAC with a secret key plus a per-run salt. The same run yields the same token.

A different run yields a different token. That is deliberate. It blocks accidental joins across runs.

A minimal redactor

Illustrative and unexecuted. Read it before you run it.

# redact.py - illustrative, unexecuted
import hashlib
import hmac
import json
import os
import re

SECRET = os.environ['TRACE_HMAC_KEY'].encode()  # never stored in traces

STRUCTURAL = {'span_id', 'parent_id', 'seq', 'name', 'kind', 'status', 'tool'}
SCRUB = {'args', 'output', 'error', 'prompt'}

EMAIL = re.compile(r'[\w.+-]+@[\w-]+\.[\w.]+')
BEARER = re.compile(r'(?i)bearer\s+[A-Za-z0-9._-]{12,}')


def pseudonym(salt: bytes, value: str) -> str:
    mac = hmac.new(SECRET, salt + value.encode(), hashlib.sha256)
    return 'anon_' + mac.hexdigest()[:12]


def scrub(text: str, salt: bytes) -> str:
    text = EMAIL.sub(lambda m: pseudonym(salt, m.group(0)), text)
    return BEARER.sub(lambda m: pseudonym(salt, m.group(0)), text)


def redact_span(span: dict, salt: bytes) -> dict:
    out = {}
    for key, value in span.items():
        if key in STRUCTURAL:
            out[key] = value
        elif key in SCRUB and isinstance(value, str):
            out[key] = scrub(value, salt)
        # unknown keys are dropped on purpose
    return out


def redact_trace(src_path: str, dst_path: str, salt: bytes) -> None:
    tmp = dst_path + '.tmp'
    with open(src_path) as src, open(tmp, 'w') as dst:
        for line in src:
            if line.strip():
                span = json.loads(line)
                dst.write(json.dumps(redact_span(span, salt)) + '\n')
    os.replace(tmp, dst_path)  # atomic hand-off

Two design choices are worth naming. Unknown keys are dropped, which is the deny-by-default rule. That also drops fields you add later, so your tests must catch it.

Writes go to a temp file, then os.replace. A crashed export leaves no half-redacted bundle for someone to share by mistake.

Verify the diff still aligns

The property you want is narrow and checkable. Redaction must not change which field paths differ between a good run and a bad one.

# tests/test_redaction_shape.py - illustrative, unexecuted
import json


def divergent_paths(a: dict, b: dict, prefix: str = '') -> set:
    paths = set()
    for key in set(a) | set(b):
        path = prefix + key
        va, vb = a.get(key), b.get(key)
        if isinstance(va, dict) and isinstance(vb, dict):
            paths |= divergent_paths(va, vb, path + '.')
        elif va != vb:
            paths.add(path)
    return paths


def test_redaction_preserves_divergence():
    with open('fixtures/span_good.json') as f:
        good = json.load(f)
    with open('fixtures/span_bad.json') as f:
        bad = json.load(f)
    salt = b'run-2026-11-10'
    assert divergent_paths(good, bad) == divergent_paths(
        redact_span(good, salt), redact_span(bad, salt)
    )

pytest -q tests/test_redaction_shape.py

Loop that assertion over every span pair in the run, not one fixture. What the test proves: your redactor does not hide or invent divergence points. What it does not prove: that nothing sensitive survived. That needs a canary.

The canary test should fail first

# tests/test_canary.py - illustrative, unexecuted
import json


def test_no_canary_survives():
    span = {'span_id': 's1', 'tool': 'http_get',
            'args': 'GET /v1/users?api_key=sk-test-CANARY-123'}
    out = json.dumps(redact_span(span, b'run-1'))
    assert 'CANARY' not in out

The pattern list above will miss formats you did not anticipate, and this fixture is one of them. Run the canary, watch it fail, add a pattern, run it again. That loop is the actual work.

Wire it into a repeatable loop

1. Capture with an allowlist of field names, not a blocklist of patterns.
2. Store raw traces behind access control. Treat them as production data.
3. Reproduce the failing run and confirm the failure is stable.
4. Localize the first divergent field path between good and bad runs.
5. Hand off an exported, redacted bundle. Run the shape test in CI on every change.

Step 5 is where teams skip verification. Make export a CI job, not a manual step someone remembers.

Limits and who should not use this

• Deletion requirements. A pseudonym is not a deletion. Regulated data needs real erasure.
• Free-form text. Model output quoting user text slips past field-level rules.
• Cross-run joins. The per-run salt blocks them by design. Use a scoped key if you need them.
• Leaked key with low-entropy values. Emails are guessable. Rotate the key immediately if it leaks.
• Small local-only traces. File permissions may be enough. Skip the pipeline.

If you cannot state what your redactor does not cover, do not share the trace.

Where cheap reruns fit

This loop needs fresh traces every time you touch the redactor. That is the narrow place where tooling access helps. MonkeyCode is an open-source project with free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Both options remove setup friction, not correctness work. Free model access means a regenerate-and-retest cycle does not begin with a billing decision. The free server option avoids provisioning a box for the export pipeline. Verify the current limits and terms on the project's official page before you plan around them.

Nothing above depends on the product. The loop runs on any model you can rerun. If you want to try it before wiring billing, the free tier is the cheapest place to start.