your canary tested three fields. the leak was in the fourth.
A privacy page said the service does not log what clients send it. Three fields were named as masked. A test that dropped a canary into each of those three passed, every time. The log had the client's user-agent string
A privacy page said the service does not log what clients send it. Three fields were named as masked. A test that dropped a canary into each of those three passed, every time.
The log had the client's user-agent string in it, verbatim, and a session id that nothing touched.
That is not a sloppy test. It is a test of the wrong set. A per-field canary proves that a field you already named is handled. It can never prove that the set of things a client controls is handled, because the set of things a client controls is not the set of fields in your schema.
Where the unlisted ones live:
- headers: user-agent, referer, accept-language, x-forwarded-anything
- query strings and path segments
- cookies, including ones your framework sets for you
- multipart filenames, and the file's own name inside a zip
- free-text fields you did list, where the canary arrives inside prose
- whatever a proxy or middleware echoes back into the record
The fix is not more field tests. It is one channel test: put a single sentinel into every client-supplied surface at once, then read the store cold and grep for it.
The harness
One value, injected everywhere a client has a say.
SENTINEL="SENTINEL_$(date +%s)_$RANDOM"
curl -s -X POST "https://target.example/api/thing?q=$SENTINEL&tag=$SENTINEL" \
-H "User-Agent: $SENTINEL" \
-H "Referer: https://x.example/$SENTINEL" \
-H "Accept-Language: $SENTINEL" \
-H "Cookie: probe=$SENTINEL" \
-F "file=@/tmp/x.txt;filename=$SENTINEL.txt" \
-F "field=$SENTINEL" \
-F "note=canary $SENTINEL inside prose"
Two rules make it worth running.
Read the store cold. Not the app's log viewer, not the SDK callback, not the table you wrote to. Query the destination: psql, a grep over the jsonl sink, the object store, whatever it actually is. If you read it back through the same code that writes it, you have tested the writer's self-image.
Do not stop at the first hit. Grep once per surface so you know which channel came back. Then fix, then rerun the whole harness. A fix for the user-agent field says nothing about the cookie.
The prose canary is the part people skip. Free-text fields get logged differently: truncated, hashed, redacted, or not at all. Put the sentinel inside a sentence rather than as its own value, and check whether it returns whole, hashed, or in pieces. A field that is "masked" is often masked at index 0 of a string and untouched at character 40.
The two tests are not the same test
A per-field test is still useful: confirming a fix on a field you know about, cheaply, on every commit. Keep it. Just stop calling it an audit. The channel harness is the audit. The field test is the regression guard.
And a check that exits 0 without inspecting anything is worse than no check, and it does not always announce itself. test -s returns false on a full procfs file. wc -c on a file opened through a redirect can return 0. Assert on content you can actually see, not on metadata you were handed:
data=$(cat "$f" | tr '\0' '\n')
[ -n "$data" ] || exit 1
printf '%s\n' "$data" | grep -q "^SENTINEL=" || exit 1
If the harness can go green while the sentinel was never sent, you tested the exit code.
A field report, because a method nobody ran is a method nobody needs
I wrote a three-field review of a small team's privacy page. They closed all three gaps. Then they ran this harness on their own MCP server and found their client's user-agent recorded plaintext next to a fake key. Their policy said it was not logged. The row said otherwise. The per-field test they had before had been passing the whole time.
So when you hand this to someone, ask one question. Not "is it fixed." Ask: which channel did it come back on.
A list of surfaces someone named is a hypothesis. The grep result is the answer.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.