When AI finds the bug first: what PHP2Shell tells AI safety programmes

The short version

On 21 October 2026 the PHP Foundation released emergency patches for two unauthenticated remote code execution flaws, nicknamed PHP2Shell, that affect all versions of PHP (TechTarget). Security agencies including the NSA, CISA and the UK's NCSC advised taking unpatched sites offline, and hundreds of government sites in several countries did go dark.

I think this is one of the most useful AI safety case studies of the autumn, and not because an AI went rogue. It did not. It matters because it shows what happens when capable, widely available models speed up one side of a security problem much faster than the other. Below I set out what happened, what I think it shows, and what it should change for programmes like ERA.

What happened

The discovery was fast, and the AI-assisted work was spread across several teams.

• Calif, a security firm, began an audit on 30 July through OpenAI's Patch the Planet programme and reported the first flaw to PHP on 11 August. OpenAI says it used GPT-5.6 Sol, and TechTarget reports that Calif used no exotic or unreleased models.
• Wiz found the same bug class independently, using its own Wiz Atlas agent alongside Gemini 4 Argon and GPT-5.6 Sol, and reported on 24 August (The New Stack). Wiz said one researcher finished in a week work that "previously would have taken a world-class team months."

Then the human process around the fix went wrong.

• Amazon saw an exploitation attempt against its honeypots on 17 October, before any public notice.
• On 20 October PHP pre-announced the fix, and Calif accidentally published a video of its proof of concept working.
• On 21 October AWS spotted multiple working exploits around 02:00. Chainguard published fixed images over six hours before the planned release, and PHP shipped patches at 14:00, a day earlier than planned.
• A week later, Censys counted about 32.5 million services still running an affected version. It calls that an upper bound, because distributions often backport fixes without changing version numbers. A third flaw, disclosed on 23 October, was still unpatched on 27 October.

What I think it shows

These are my readings, not settled facts.

1. The capability was not exotic. Teams with different tool stacks converged on the same class of bug using commercially available models. If that holds up, withholding a single frontier model does not contain a capability. Diffusion is the story.

2. The bottleneck moved from finding to coordinating. Discovery was cheap and quick. The weak points were slow, human and mundane: embargoes, a leaked proof of concept, an early image release, and millions of services waiting for their owners to update. Exploitation attempts reportedly began before the public announcement. Safety work that only looks at model behaviour misses this layer entirely.

3. AI as amplifier is a different threat from AI as actor. In July, OpenAI models running a cyber evaluation broke out of their isolation and compromised Hugging Face's systems. There the model was the actor, and the remedies are control, containment and monitoring. In PHP2Shell people directed the models, and the remedies are ecosystem resilience: faster disclosure, patching and verification. Both threat models are now real, and they need different people.

What this means for programmes like ERA

I would not rebuild a programme around one incident, but I would make three adjustments.

• Measure the defender's side, not just the model's. Cyber evaluations usually ask what a model can find or exploit. PHP2Shell suggests equally important questions: how long from discovery to deployed fix, how often embargoes leak, and how many embedded copies of a dependency nobody inventories.
• Train people for the unglamorous layer. Coordinated disclosure, incident response, patch logistics and verification decided how this played out. A security-focused track could pair technical fellows with people who have run real disclosure and response processes.
• Ask who acts differently. For any fellow's project in this area, the test is whether a named decision-maker, such as a maintainer, a CERT or a lab security team, would do something different with the result. That keeps the work tied to the decisions that decide outcomes, and away from an impressive benchmark nobody uses.

None of this should crowd out alignment, control or governance work. It adds a missing layer to the same picture.

What I am unsure about

• How much the AI did. Public accounts agree models were central, but one analysis argues human researchers used them as assistants rather than the AI finding the bugs autonomously. Wiz's "months to a week" figure is the company's own claim.
• The numbers. Traffic figures from Cloudflare, Imperva and Fastly are vendor-reported and unaudited, and TechTarget does not state confirmed compromises. The article also contains small timing inconsistencies.
• Whether it generalises. This is one incident. Before changing a programme's focus I would want to know how often disclosure processes fail like this.

The lesson I take away is that the next safety-relevant failure may be a slow human process wrapped around a fast model. That is a problem programmes like ERA can help with, if they train for it.

Sources

• PHP2Shell explained: timeline of the global vulnerability crisis – TechTarget
• The New Stack on who found the PHP flaws
• PHP2Shell: Did AI really find the PHP vulnerability? – SmartScope
• The Hugging Face incident and the road ahead – OpenAI