Audit a Trace Slice Before a Model Explains the 500

Checkout returned 500. The next message in the thread was the dangerous one: paste the trace, let the model explain it. I built that thread as a fixture, not from a customer ticket. Line three still has a session cookie and reset_token in the query string.

That is the trust-boundary violation. The model is not on the request path. The paste is. Why would I hand a session cookie to a system whose job is to echo, summarize, and sometimes retain?

I am not claiming a live incident, a bounty, or a vendor finding. What follows is an unexecuted template: a pre-send gate, two fixtures, and the failure evidence I want CI to demand. I have not run it against a production model, and I will not pretend I have.

Where the boundary actually sits

A trace slice has more than one owner. Forget the model for a second.

1. The app emitted the line.
2. The log pipeline stored it.
3. A human copied a window of it.
4. A model endpoint, or a helper server, received the copy.

Steps 1 and 2 can stay inside your trust zone. Step 4 often does not. A hosted model is someone else's process. A server you stood up because it was free is still a second machine, with disk, logs, and an admin path you may not have reviewed.

So the invariant is small. No secret-bearing field crosses step 4. Not in the prompt. Not in a file upload. Not in a "debug context" attachment.

Does a redaction regex make the model safe? No. It only stops the obvious paste. That is still the gate I want in front of the clipboard.

app  ->  log store  ->  human slice  ->  [pre-send gate]
                                          | deny: stop
                                          | allow: model or assistant server
raw slice stays in the incident store. It does not ride along.

Logs are the messy part. An access line, a span attribute, and a copied shell scrollback can all carry the same cookie under different names. If I only scrub the prompt box, the attachment still leaves. The gate has to see the bytes that would actually be sent.

What I refuse to send

I keep a deny list, not a hope list. If a field is on it, the slice does not leave.

• Authorization, Proxy-Authorization, Cookie, Set-Cookie
• Query keys such as token, reset_token, code, password, api_key
• Connection strings (postgres://, mysql://, mongodb://, redis://)
• Cloud key shapes (AKIA, ASIA) and PEM private-key blocks
• Internal hostnames I have not explicitly allow-listed for that ticket

A stack trace without those fields is usually fine. A stack trace with a request dump is not. Would you paste the same bytes into a public gist? If the answer is no, the model does not get them either.

Headers are the obvious leak. Query strings are the one people defend. "It is just a reset link." It is a bearer credential with a question mark in front of it. Treat it that way.

Shell history is the third copy. curl -v output, kubectl describe, and a pasted .env diff are not "context." They are credential stores with worse retention than your vault. I do not attach them. I attach the redacted symptom: status, route, exception type, and the line number.

The fixture

Pinned assumptions, so this does not drift into folklore:

• Language: Python 3.12, standard library only. No package pin, because there is no package.
• Input: one JSON object, UTF-8, capped at 64 KiB in this template.
• Contract: exit 0 only when the slice is already redacted; exit 2 on a hit; never print the secret value.
• Status: unexecuted template. Expected evidence is specified below, not observed.

#!/usr/bin/env python3
"""pre_send_gate.py — unexecuted template. Not a scan result."""
import json, re, sys

DENY = re.compile(
    r"(?i)(authorization|proxy-authorization|cookie|set-cookie)\s*[:=]"
    r"|((?:reset_)?token|api_key|password|code)\s*[=:]\s*[^&\s]{6,}"
    r"|postgres(?:ql)?://\S+|mysql://\S+|mongodb(\+srv)?://\S+|redis://\S+"
    r"|\bAKIA[0-9A-Z]{16}\b|\bASIA[0-9A-Z]{16}\b"
    r"|-----BEGIN [A-Z ]*PRIVATE KEY-----"
)

def main() -> int:
    raw = sys.stdin.read(65536)
    if sys.stdin.read(1):
        print("reject: oversize", file=sys.stderr)
        return 2
    try:
        doc = json.loads(raw)
    except json.JSONDecodeError:
        print("reject: not-json", file=sys.stderr)
        return 2
    blob = json.dumps(doc, ensure_ascii=False)
    if DENY.search(blob):
        print("reject: secret-shaped field", file=sys.stderr)
        return 2
    print("allow: no deny-pattern hit")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Short script. Narrow job. It does not call a model, and it must not grow a client "for convenience."

1. Negative fixture

Save this as fixtures/negative.json. It should fail closed. The values are synthetic. Do not replace them with real ones to "make the test better."

{
  "path": "/checkout",
  "status": 500,
  "request_headers": "Cookie: session=synthetic-session; Authorization: Bearer synthetic-demo",
  "query": "reset_token=synthetic-secret-value"
}

Expected, not observed: python3 pre_send_gate.py < fixtures/negative.json exits 2 and prints reject: secret-shaped field on stderr. The secret itself must not appear on stdout. If your run prints the cookie, the gate is an exfiltrator. Delete that print before you debug anything else.

2. Positive fixture

Save this as fixtures/positive.json. Same incident shape. Secret-shaped fields are dropped, not masked after the colon. This template treats Cookie: itself as a hit, so Cookie: <REDACTED> still fails. Good. Fail closed on the field, not on your confidence in the mask.

{
  "path": "/checkout",
  "status": 500,
  "headers_present": ["cookie", "authorization"],
  "query": "<REDACTED>",
  "error": "NullPointerException at CheckoutService.java:88"
}

Expected, not observed: exit 0 and allow: no deny-pattern hit. If the positive fixture still fails, the slice is not clean, or the pattern is broader than you thought. Do not weaken the regex to force a pass. That is how gates rot.

headers_present is the compromise I will allow. The model can know a cookie header existed. It cannot know the cookie. If you need the model to see a header name, put the name in a list. Never leave the colon-form in the blob.

3. Keep the model unreachable on failure

python3 pre_send_gate.py < fixtures/negative.json
test $? -eq 2 || echo "gate failed open"
python3 pre_send_gate.py < fixtures/positive.json && echo "human may paste this slice"

No model URL belongs in the failing branch. If I cannot show the exit code, I do not have a gate. I have a comment in a README.

Run this on Python 3.12 and keep the stderr next to the commit. A screenshot of a chat reply is not evidence. The exit code is.

Prevent, detect, recover

Rotation is not optional once a negative slice has crossed the boundary. I cannot audit a recipient's retention from a chat transcript. Can you? Then do not bet the session on it.

Detect is where teams get lazy. They run the positive fixture, see allow, and call the control done. The negative fixture is the control. A gate that has never failed in CI has never been shown to work.

Where a free model and a free server fit

This is where a product can sit in the workflow, and also where a free tier makes a bad paste feel cheap.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator describes free model access and a free server option for MonkeyCode, an open-source AI development platform. I am not pinning a token quota, a model id, a hardware size, or a duration. Those figures move, and I do not have a primary source in this draft. A stale allotment would be worse than none.

Read the placement this way.

A free model is still an egress destination. Free does not mean inside your VPC. A free server is still a host. If you self-host the assistant there, you moved the process closer. You did not delete the boundary. Disk, shell history, and container logs on that box can retain the same cookie you refused to paste into a hosted chat.

The workflow does not change with the price tag.

1. Export the smallest slice that explains the 500.
2. Run the gate. A negative result stops the workflow.
3. Only the positive slice may enter a prompt, whether the endpoint is free, paid, or on a server you stood up this morning.
4. Keep the raw slice in the incident store. It does not ride along as extra context.

Would I send the negative fixture to a free tier just to see what the model says? No. The free tier is where casual paste happens, because the cost signal is gone. The security signal should not leave with it.

If you want to try that platform, read the live terms for the free model access and the free server option, then point only a redacted slice at it.

Limitations, and who should walk away

This regex is a fixture, not a DLP product. It misses split secrets, homoglyphs, values logged under a name I did not list, and secrets already copied into a stack message. It will also false-positive on docs that quote an Authorization: example, and the bare code alternative is deliberately broad. Fail closed anyway. A false positive costs a rewrite. A false negative costs a session.

Do not use this approach if any of these are true.

• You handle regulated data, and even a redacted trace may not leave the zone.
• You need attested coverage, key custody, or a vendor questionnaire. Forty lines of Python are not that.
• Your team will bypass the gate when the model "needs more context." The bypass is the incident.
• You came for a WAF rule, a deployment guide, or a mass-assignment fix. Different boundary.

I also did not execute the commands in this draft. If you adopt the fixture, run both files yourself and keep the exit codes next to the commit. An unrun template is a proposal. A captured exit code is evidence.

There is a second limitation I care about more than the regex. The gate does not know your business. An employee id, a medical record number, or an unlisted internal hostname can be sensitive and still sail through. Add those patterns locally. Do not wait for a generic list to grow a conscience.

What belongs in CI?

Which invariant belongs in CI, and which layer should enforce it?

I would put the fixture pair in CI. Negative must exit 2. Positive must exit 0. The job must not hold a model credential at all. The log pipeline, not the model client, should strip secret-shaped fields before a human can copy a window. CI proves the gate still fails closed. The pipeline proves the secret never became copyable.

If your CI job needs a live model key to "test the prompt," the boundary is already wrong. Fix that before you ask any model why checkout returned 500.