Classify the Field, Not the File: A Redaction Gate for Free Model Tiers

Here is a request body I keep finding in agent logs. Nothing exotic — a stack trace, a config dump, a database URL with the password still attached:

{"role":"user","content":"Why does this fail? postgres://app:[email protected]:5432/prod — timeout after 30s"}

Every file in that repo passed review. The leak did not come from a file. It came from a field — one string inside a payload that nobody classified. So the invariant I want to enforce is narrow and testable: no outbound string containing a credential-shaped value leaves the machine, even when the file it came from is allowed to leave.

Can a gate do that without turning your context into useless mush?

The trust boundaries I actually care about

When you point an agent at a free model tier and a free server, four boundaries matter more than the marketing page:

• Repo → agent. Untrusted content (READMEs, dependency docs, issue text) becomes instructions.
• Agent → request body. Trusted-ish files get serialized, and serialization flattens classification. A .env reader and a README reader produce the same JSON field.
• Request body → provider. Once it is sent, your retention story is the provider's, not yours.
• Response → your shell. Completions are untrusted input the moment anything executes them.

This article only hardens boundary two, because it is the cheapest to test and the one most teams skip. Boundaries one and four need separate gates; I have written about those elsewhere and will not pretend this artifact covers them.

Step 1 — Enumerate egress channels before writing rules

Do not start with regex. Start with a list, because the channels differ in shape:

1. Chat/completion request bodies (your typed prompt).
2. Files attached or read by a tool and inlined verbatim.
3. Tool outputs: docker logs, env, stack traces, HTTP errors.
4. Filenames and paths inside those outputs.
5. Anything you paste into a review comment that a bot then forwards.

Channel 3 is where the DSN above came from. Channel 4 is the sneaky one: ~/.aws/prod-credentials as a path is already a disclosure.

Step 2 — Write rules for structure, not for vibes

Pattern rules catch things entropy cannot. Entropy catches things patterns miss. You need both, and you need to know which one is load-bearing.

# payload_gate.py
import hashlib, hmac, os, re
from math import log2

KEY = bytes.fromhex(os.environ["REDACT_KEY"])  # 32 bytes, never leaves the host

RULES = {
    "aws_key":     re.compile(r"\b(?:AKIA|ASIA)[0-9A-Z]{16}\b"),
    "conn_string": re.compile(r"\b(?:postgres|mysql|mongodb(?:\+srv)?|redis)://[^\s\"']+"),
    "bearer":      re.compile(r"\bBearer\s+[A-Za-z0-9._~+/\-]{20,}=*"),
    "private_key": re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"),
    "jwt":         re.compile(r"\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}"),
}

TOKEN_RX = re.compile(r"[A-Za-z0-9_+/=\-]{24,}")
# git sha, sha256 digest, uuid: high-looking but not secret
SAFE_RX = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64}|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$")

def entropy(s: str) -> float:
    if not s:
        return 0.0
    counts = {}
    for c in s:
        counts[c] = counts.get(c, 0) + 1
    return -sum((n / len(s)) * log2(n / len(s)) for n in counts.values())

def placeholder(kind: str, value: str) -> str:
    tag = hmac.new(KEY, value.encode(), hashlib.sha256).hexdigest()[:10]
    return f"[[redacted:{kind}:{tag}]]"

def scrub(text: str):
    hits, out = [], text
    for kind, rx in RULES.items():
        def _sub(m, kind=kind):
            hits.append({"rule": kind, "len": len(m.group(0))})
            return placeholder(kind, m.group(0))
        out = rx.sub(_sub, out)
    def _sub_entropy(m):
        tok = m.group(0)
        if SAFE_RX.match(tok) or entropy(tok) < 4.0:
            return tok
        hits.append({"rule": "high_entropy", "len": len(tok)})
        return placeholder("high_entropy", tok)
    return TOKEN_RX.sub(_sub_entropy, out), hits

Two design decisions are worth defending. First, placeholders are deterministic per value, so if the same secret appears five times the model still sees one consistent token and its reasoning does not collapse into five unrelated entities. Second, the mapping key is HMAC-local, which means you can map back for debugging while the log itself stores nothing reversible.

Step 3 — Fixtures, both directions

A redaction gate with only positive tests is a regex with a good publicist. You need to prove it does not fire on legitimate content, or your team will disable it in week two.

# test_payload_gate.py
import pytest
from payload_gate import scrub

MUST_REDACT = [
    ("key=AKIAIOSFODNN7EXAMPLE", "aws_key"),
    ("db=postgres://app:[email protected]:5432/prod", "conn_string"),
    ("Authorization: Bearer abcdefghijklmnopqrstuvwxyz012345", "bearer"),
    ("hoA9sK2mQ8xZ1pL4vN7bT0rY3wE6uI5oP2aS", "high_entropy"),
]

MUST_PASS = [
    "commit 9f2c1b8a4d5e6f708192a3b4c5d6e7f809a1b2c3",
    "id 550e8400-e29b-41d4-a716-446655440000",
    "getUserBySessionId(null)",
]

@pytest.mark.parametrize("payload,rule", MUST_REDACT)
def test_redacts(payload, rule):
    out, hits = scrub(payload)
    assert rule in {h["rule"] for h in hits}, hits
    assert "hunter2" not in out
    assert "AKIAIOSFODNN7EXAMPLE" not in out

@pytest.mark.parametrize("payload", MUST_PASS)
def test_passes(payload):
    out, hits = scrub(payload)
    assert out == payload and hits == [], hits

def test_same_secret_same_token():
    a, _ = scrub("x=AKIAIOSFODNN7EXAMPLE")
    b, _ = scrub("y=AKIAIOSFODNN7EXAMPLE")
    assert a.split("=")[1] == b.split("=")[1]

Run it with REDACT_KEY=$(openssl rand -hex 32) pytest -q test_payload_gate.py. The expected evidence is 7 passed. If test_passes fails on your repo's real strings, fix the gate, not the fixture.

Step 4 — Put it in front of the socket, not in the prompt

A redaction step a developer can skip is not a control. Wrap the client:

def send(messages):
    clean, hits = [], []
    for m in messages:
        text, h = scrub(m["content"])
        clean.append({**m, "content": text}); hits += h
    ledger_append({"ts": time.time(), "hits": hits})  # counts only, no values
    return client.chat(messages=clean)  # endpoint from MODEL_BASE_URL

One caveat I will not paper over: this assumes a chat-completions-shaped request. Whether any given free tier accepts that shape, and what its limits are, has to come from that provider's current docs — I am not asserting a protocol for you.

Step 5 — The decision table your reviewer actually needs

Limitations, honestly

Hex-encoded keys cap at 4.0 bits per character, so the entropy pass will never catch them — that is why the pattern rules exist and why entropy is the fallback, not the primary. The gate also cannot see secrets split across fields, base64-encoded inside an image, or encoded in a variable name. It does not classify meaning; it classifies shape. And if you cannot hold REDACT_KEY on the host, the deterministic-placeholder design buys you nothing.

Who should skip this? Teams whose credentials never enter the repo (vault-only, short-lived tokens) get little from it. Teams needing compliance-grade DLP need a real classifier and a review trail, not 60 lines of Python.

Where to run it cheaply

You want a live endpoint to run Step 3 against, because fixture-only testing hides serialization bugs. Free tiers are useful here precisely because they are disposable: send scrubbed payloads, watch the ledger, throw the workspace away. The MonkeyCode project is one such option — it is open source, and its operator states that it offers free model access (a 10M-token allowance) plus a free server option; check the project page for the current terms rather than trusting a number in a blog post, mine included.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

So: which invariant belongs in your pre-send hook, and which one belongs in the provider's contract? My answer is that redaction is yours — you cannot audit a boundary you do not enforce — and retention is theirs, which is why you should write down what you assume before the first request goes out.