A regex DLP layer for an LLM gateway: blocking keys, masking IDs, and what to do with chat history

LLM gateways are becoming standard infrastructure: one service sits between your team and the model providers, handling auth, rate limits and logging. But most write-ups stop at forwarding. The interesting part is what happens before a request leaves your network — the data-loss-prevention (DLP) layer.

This post walks through the design of a regex-based DLP layer I built for a self-hosted LLM gateway (MIT, source linked at the end). It's deliberately simple: no ML classifier, no NER model — just a rule engine that has to be fast, predictable and hot-reloadable.

The rule model

Every rule is three things: a regex, an action, and a label.

{ "id": "block-api-keys", "pattern": "sk-[a-zA-Z0-9]{20,}", "action": "block", "label": "API key" }
{ "id": "mask-credit-card", "pattern": "\\d{16,19}", "action": "mask", "label": "Card number" }

Three actions, in increasing severity:

• log — record the hit, let the request through. Good for tuning: run in log-only mode for a week, see what your false-positive rate looks like before you ever block anything.
• mask — replace the match but keep the request alive. For IDs and card numbers we keep the first and last 4 characters: 6222 0211 **** 1234. Enough to recognize which card it was in an audit, not enough to use.
• block — reject the request outright with a 4xx. Used for credentials, because a masked API key is still a leaked key: masking sk-abc...xyz just tells an attacker which characters to brute-force around.

The engine caps rules at 50. That's not a technical limitation — it's a design statement. If your DLP policy needs more than 50 regexes, the problem is your policy, and no engine will save you.

Only scan the last user message

The naive implementation scans every message in the messages array on every request. That's O(history) per request, and chat history grows.

But here's the thing: every historical message was already scanned when it was sent. Re-scanning it adds latency and creates a weird failure mode — a user sends a message today, the policy tightens tomorrow, and suddenly their entire conversation stops working.

So the gateway scans only the last user message. History hits are handled differently: if a historical message matches a block rule, it's replaced wholesale with a [redacted] placeholder rather than trying surgical masking.

Why wholesale? Because surgical redaction of history gives a false sense of precision — you don't know which tokens the model will actually attend to, and partial redaction of a known leak is worse than honest replacement.

Hot reload, the boring way

Rules live in the gateway config. The scanner reads the current config on every request — no in-memory cache to invalidate, no reload endpoint to call, no "did the workers pick it up" doubt.

Is re-reading a JSON file per request wasteful? At the scale of one team or one company, the config read is nanoseconds against a multi-second LLM call. Boring wins.

What regex DLP can't do

Be honest about the limits:

1. Semantic leakage. "Our CTO's password is the name of his dog followed by 123" sails through every regex. Blocking that requires classification, not pattern matching.
2. Encodings. Base64-encoded keys, homoglyphs, deliberate spacing (s k - a b c). Some are patchable with more regexes; some are a rabbit hole.
3. Structured data beyond the patterns you wrote. This is a allowlist-of-known-bad approach, not a guarantee.

The roadmap answer is a classifier stage after the regex stage — regex catches the cheap 95% with zero false positives, a model catches semantics with a human-review queue. But shipping regex-only DLP is still strictly better than shipping nothing: the default rules (API keys, ID numbers, card numbers, phone numbers) catch what actually leaks in practice.

Try it

The gateway is open source (MIT): a single Node.js process with built-in SQLite, no external dependencies. The DLP rules are configurable per deployment, and the whole thing — auth, model catalog, plugin allowlists, billing, audit logs with daily tamper-evident anchors — is one node gateway.mjs away.

• Live demo (no signup, admin console fully clickable): https://fffly.com/demo
• Source: https://github.com/mafeis/dsh-enterprise

Feedback welcome — especially from anyone who's run DLP in production and has scars to share.