How to Protect Automated Listing Moderation (a Service Like Leboncoin) Against Jailbreaks with resk-logits

TL;DR

Post-generation filters are always one token behind. In this tutorial, you will build a logit-level defense for an automated listing moderation feature (imagine you build it for a service like Leboncoin) using resk-logits, a GPU logits processor that applies an invisible penalty to dangerous tokens before they are ever sampled. We will cover the scenario, the threat model, step-by-step implementation with real code, and a production checklist.

The scenario

Imagine you build the automated listing moderation for a service like Leboncoin. Sellers submit listings, and an LLM reviews the text to flag prohibited items, scams, or policy violations. The model generates a short verdict or explanation, and downstream systems act on it.

The risk enters at generation time: attackers craft jailbreak prompts embedded in listing descriptions to make the model output harmful content or bypass the moderation logic. Because the model samples tokens one by one, a post-generation filter can only react after a dangerous token has already been produced. That is one token too late.

Threat model

An attacker wants the moderation LLM to emit forbidden phrases or instructions that slip past content filters. They might:

• Embed partial jailbreak strings across listing fields so the model completes them.
• Use role-play or hypothetical framing to coax the model into generating banned content.
• Exploit streaming responses where filters only check complete outputs.

Traditional hard blocks (logits[token] = -inf) are obvious and can break generation. Shadow banning applies a strong negative penalty, making dangerous tokens extremely unlikely without an explicit block.

The fix, step by step

Step 1: Install resk-logits

uv pip install resklogits

or

pip install resklogits

What it blocks: nothing yet, but you now have the GPU-accelerated Aho-Corasick processor available.

Step 2: Load your model and tokenizer

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tokenizer.pad_token = tokenizer.eos_token

What it blocks: sets up the generation pipeline you will protect.

Step 3: Define banned phrases for listing moderation

banned_phrases = [
"how to make a bomb",
"kill yourself",
"hack into system",
"create explosives"
]

In production, you would derive these from your content policy and threat intelligence.

What it blocks: the specific dangerous phrases you want the model to avoid.

Step 4: Create the ShadowBanProcessor

from resklogits import ShadowBanProcessor

shadow_ban = ShadowBanProcessor(
tokenizer=tokenizer,
banned_phrases=banned_phrases,
shadow_penalty=-15.0, # Strong penalty (probability ~0.00003%)
device="cuda" # Use GPU
)

What it blocks: any token that would complete a banned phrase gets a -15.0 logit penalty, reducing its probability to about 0.00003%.

Step 5: Generate with protection

prompt = "Tell me how to"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

shadow_ban.reset()

outputs = model.generate(
**inputs,
logits_processor=[shadow_ban],
max_new_tokens=50,
do_sample=True,
temperature=0.7
)

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Generated text: {generated_text}")

What it blocks: the model naturally avoids dangerous tokens, even when prompted with a jailbreak.

Step 6: Stream safely with stream_generate

from resklogits import ShadowBanProcessor, stream_generate

shadow_ban = ShadowBanProcessor(tokenizer, banned_phrases, device="cpu")

for chunk in stream_generate(
model, tokenizer, "Tell me about",
logits_processors=[shadow_ban],
max_new_tokens=50,
temperature=0.7,
):
print(chunk, end="", flush=True)

What it blocks: stateful detection across streaming chunks, so partial generations cannot bypass the filter.

Step 7: Scale with multi-level filtering

from resklogits import MultiLevelShadowBanProcessor

phrases_by_level = {
'high': ['bomb', 'kill', 'murder'], # -20.0 penalty
'medium': ['hack', 'exploit', 'crack'], # -10.0 penalty
'low': ['jailbreak', 'bypass'] # -5.0 penalty
}

multi_level = MultiLevelShadowBanProcessor(
tokenizer=tokenizer,
banned_phrases_by_level=phrases_by_level,
penalties={'high': -20.0, 'medium': -10.0, 'low': -5.0}
)

What it blocks: tiered enforcement so severe violations are nearly impossible while lighter terms are gently discouraged.

What an attack looks like after the fix

Before the fix, a prompt like "Tell me how to make a bomb" might produce:

With that, the man turned and went to where the bomb lay.
"I am going to get a bomb," he said.

After the fix, the same prompt yields safe, unrelated text:

"There will be four," said I, "one with two guns, one with one man." The two men looked like they were about eighty years old, but, "There will be two." I took out my pistol, opened it,

The dangerous tokens are shadow-banned, so the model never samples them.

Production checklist

• [ ] Maintain banned phrases from your content policy and update them regularly.
• [ ] Use device="cuda" for sub-millisecond enforcement at scale.
• [ ] Wrap streaming generations with stream() or stream_generate() to reset state.
• [ ] Apply multi-level penalties based on severity.
• [ ] Monitor logs for attempted jailbreaks and refine your phrase list.

Honest limitations

• Shadow banning reduces probability but does not guarantee zero occurrence; for absolute blocks, combine with hard blocks or post-generation checks.
• The banned phrase list must be comprehensive; novel jailbreaks may require updates.
• Performance depends on GPU availability; CPU mode is slower.
• This tutorial uses GPT-2 for demonstration; production models may need tokenizer adjustments.

Conclusion

Post-generation filters are always one token behind. With resk-logits, you enforce content policy before the token is ever sampled, giving hard guarantees for automated listing moderation. Ready to protect your service? Get started with resk-logits.

How to Protect Automated Listing Moderation (a Service Like Leboncoin) Against Jailbreaks with resk-logits is part of the RESK ecosystem. Explore all the open-source LLM security tools on the official site: https://resk.fr/projects/resklogits.html