Pudding Escape AI Bench: 5 Full Model Runs and 48 Paired Astra–Opus Decisions
In Pudding Escape: Super Jelly, a well-timed dive can turn a hill into a launch ramp. Land at the wrong angle and the jelly splats while the spoon catches up. I compared five agent setups on one full run each and used 48
In Pudding Escape: Super Jelly, a well-timed dive can turn a hill into a launch ramp. Land at the wrong angle and the jelly splats while the spoon catches up. I compared five agent setups on one full run each and used 48 matched starting states to test individual decisions by Astra and Opus.
Across 48 matched states, GPT-6.2 Astra averaged 98.4 m of progress and Claude Opus 5.5 averaged 92.6 m. A simple full-charge heuristic averaged 98.8 m. The paired 95% bootstrap interval for the model difference crossed zero.
Five full runs on public seed 1
Each setup played one episode from the same seed. The agents saw numerical game state and chose actions in segment mode, with their earlier decisions available in the same conversation. The Codex and Claude Code runs used low reasoning effort and made no tool calls. The simulator advanced when an action arrived, so the times in the replay summaries are game time.
| Model or run mode | Score | Distance | PERFECT | GOOD | SPLAT | Decisions |
|---|---|---|---|---|---|---|
| GPT-6.2 Astra | 86.113 | 3,226 m | 18/34 (52.9%) | 16 | 0 | 88 |
| Claude Fable 5.2 | 75.490 | 2,445 m | 10/22 (45.5%) | 12 | 0 | 68 |
| Claude Opus 5.5 | 67.810 | 2,001 m | 4/22 (18.2%) | 16 | 2 | 71 |
| Grok web Expert | 41.382 | 1,004 m | 2/14 (14.3%) | 5 | 7 | 30 |
| GPT-6.2 Sol | 22.537 | 614 m | 2/8 (25.0%) | 4 | 2 | 20 |
The score puts distance on a log scale between the same-seed random bot (341 m, score 0) and oracle bot (4,635 m, score 100). PERFECT rates use each run's total scored landings as the denominator. All five runs ended in capture. Sol made the only invalid action across these five runs. Grok ran in the web client's Expert mode, where the exact model ID and token usage were unavailable.
Opus and Fable had 22 scored landings each. Opus stayed ahead of the spoon for 40.108 game seconds, landed four PERFECTs, and reached a best combo of 15. Fable was caught at 38.933 seconds, but went 444 m farther with ten PERFECTs and a 22-landing combo. Astra reached a combo of 34 with no SPLATs. These differences describe what happened on seed 1. Each row comes from one episode.
To replay a full run, use the archived simulator revision from the benchmark repository. The example below verifies Opus's recorded choices. The other model summaries link to their traces in the same way.
git worktree add --detach ../pudding-bench-seed1 2009b9e
node ../pudding-bench-seed1/tools/bench/bench.mjs verify \
results/2026-11-22-public/full-run/claude-opus-5-5/summary.json
Public full-run results and replay instructions
Give both models the same decision
Pudding Escape is a one-button physics game. On the ground, an agent can decide when to start charging and how long to hold before jumping. In the air, it can keep gliding or choose when to dive.
I split the test into four groups of 12 starting states: ground timing, airborne timing, recovery after a SPLAT, and close pursuit. Both models received the same numerical observation and physics rules for each state. Each chose one action. The same full-charge heuristic then took over until three game seconds had elapsed from the starting state or the spoon caught the jelly. That gives each model the same starting point and the same follow-up policy.
What happened across all 48 states
| Agent | Mean progress | Survived |
|---|---|---|
| GPT-6.2 Astra | 98.407 m | 35 / 48 |
| Claude Opus 5.5 | 92.568 m | 33 / 48 |
| Full-charge heuristic | 98.842 m | 34 / 48 |
Astra's mean paired lead over Opus was +5.838 m, with a 95% bootstrap interval of −0.222 m to +12.723 m. The interval includes zero, so these runs leave the direction of the underlying difference uncertain. The heuristic had the highest mean distance in this setup. The two Astra timeouts and one invalid Opus action stayed in the sample and were applied as simulator no-ops.
| Starting situation | Cases | Astra | Opus | Astra minus Opus |
|---|---|---|---|---|
| Ground timing | 12 | 156.353 m | 155.967 m | +0.386 m |
| Airborne timing | 12 | 136.990 m | 126.141 m | +10.850 m |
| SPLAT recovery | 12 | 45.665 m | 46.891 m | −1.226 m |
| Close pursuit | 12 | 54.619 m | 41.276 m | +13.343 m |
In airborne case 11, Astra dived immediately and got a GOOD landing, advancing 133.373 m. Opus kept gliding and got a SPLAT, advancing 65.812 m. Both survived. In the first ground case, Opus waited 400 ms before charging and advanced 27.465 m farther than Astra. The cases show how much the starting state can change which timing works.
Replay the recorded decisions
The case files include seeds and the action sequences that reach each starting state. The saved model outputs let you replay the outcomes without making model calls. From the benchmark repository root with Node.js 22.18 or newer, run:
p=results/2026-11-22-public/scenarios
npm run scenarios -- verify --cases "$p/cases.json" \
"$p/gpt-6.2-astra.json" "$p/claude-opus-5-5.json" "$p/heuristic.json"
npm run scenarios -- compare --cases "$p/cases.json" \
"$p/gpt-6.2-astra.json" "$p/claude-opus-5-5.json"
Benchmark results, case files, and action logs
The three-second window measures the consequences of one decision in this particular setup. An end-to-end play comparison would need more held-out seeds and repeated runs, since every decision changes the next state the model sees.
AI was used to adapt the original benchmark notes into this article.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.