Making a Small Decision Model: Part 3, Playing Doom

When a small decision model effectively reads text tickets and selects choices from a list, expanding its input capabilities becomes a natural next step. In Part 1 and Part 2, we built and trained Gev, a 270M-parameter decision model based on Google's T5Gemma 2 backbone that answers closed text questions in 45 ms.

From the very start of this project, we chose Google's T5Gemma 2 architecture specifically because it is a Vision-Language Model (VLM). We knew going in that we wanted to work with multimodal data, and T5Gemma 2 came equipped with a built-in image reader (a SigLIP vision tower) sitting right alongside its text encoder. Just like a text tokenizer chops a sentence into words, the vision reader chops an image into a grid of small square patches and turns them into 256 visual summary tokens.

What if the input is live 35-frames-per-second gameplay from id Software's 1993 classic, Doom?

The 1993 Doom engine updates the game world 35 times a second (each step is called a game tic). Rather than asking Gev to twitch on every single frame, we ask it for a move every 4 tics (about 9 times a second, or once every ~114 ms). On each step, Gev looks at the screen and picks one of 18 controller button combinations (like "turn left", "move forward", or "fire weapon"). Under the hood, it is the exact same multiple-choice prompt we used for routing support tickets:

DOCUMENT: [Current and recent Doom screen frames]
QUESTION: Which action should the player take next?
OPTIONS: turn left <opt> turn right <opt> move forward <opt> ... fire the weapon <opt> <decide>

Figure 1: The closed-loop VizDoom inference pipeline running at 35 fps, mapping screen frames into an 18-way controller action every four game tics.

Because 1993 pixel art and fast-paced motion look nothing like English text, Doom requires fine-tuning. When you want to train a model to play a game, there are two main ways to approach it. At a high level, this is how they each work:

1. Reinforcement Learning (RL): In algorithms like PPO (Proximal Policy Optimization), the model starts knowing nothing and learns through trial-and-error rewards over millions of simulated games, gently updating weights to avoid destabilization. RL is powerful but requires massive compute clusters.
2. Behaviour Cloning (BC): In behaviour cloning, we treat gameplay as a supervised imitation learning task. We recorded 900,000 decisions from an existing skilled PPO game bot (saving screen frames and exact button presses) and trained Gev to predict the expert's action from that visual frame.

How do you turn live video into a decision problem?

A single still frame from Doom is ambiguous: a static screenshot cannot tell you whether a fireball is flying towards you, an enemy is running sideways, or you are standing still. A single still image contains zero velocity or acceleration cues.

To make intelligent decisions, the model needs motion history. We feed Gev three frames: eight game steps ago (t - 8), four steps ago (t - 4), and the current frame (t).

One thing to note though is how you pack those three frames into the vision reader. Each image gets compressed into 256 visual tokens, which led us through two designs: initially, we used a composite layout that stitched all three frames into a single image (the two older frames on top and the current frame below), keeping cost down to one 256-token pass, though cramming three frames into one image made distant enemies blurry. Later, we switched to a split layout: giving the current screen its own dedicated 256-token image and stacking the two history frames into a second 256-token image (512 visual tokens total). That gave the model a sharp view of its crosshairs right now without giving up motion history.

Figure 2: Comparing a single **composite layout* (256 tokens) against our split layout (dedicated current screen and motion history tokens, 512 tokens).*

How did we teach Gev to see pixel art without forgetting English?

Getting a 270M text model to play a 3D shooter took iteration. Initially, we started from our trained text model and fine-tuned on Doom gameplay using a single stitched image with the vision reader's weights kept completely frozen. It played surprisingly well right away, averaging 13.7 frags per match.

Looking closely at the initial logs, however, turned up two issues: the frozen vision reader was struggling to make out blurry distant sprites, and when we flipped images horizontally during training to double our data, we accidentally flipped the already-stitched canvas, which swapped the older (t - 8) and newer (t - 4) history frames on every flipped example! To fix this, we flipped each frame individually before stitching, switched to the two-image split layout, and leaned on three training habits:

1. Unlocking the upper visual layers for 1993 pixel art: The vision reader was originally trained on everyday web photos, not chunky 1993 sprites and dungeon walls. By unlocking just the top four layers of the vision reader during training (while leaving early edge-detection layers alone), the model tuned its visual filters to spot distant monsters against dark walls.
2. Mixing regular English questions into training batches so the model does not forget text: Training a small 270M model exclusively on Doom screenshots causes catastrophic forgetting, where fine-tuning on a narrow new task overwrites earlier skills. To prevent this, we mixed 128,000 regular English decision questions into the training batches alongside the Doom frames. Not only did the model play better Doom, it actually lifted its English test accuracy back up to 75.7%.
3. Hiding previous button presses 50% of the time: In fast games, repeating the button pressed four steps ago gets 10% accuracy without looking at the screen. Hiding the previous action string half the time forced Gev to watch the visual screen rather than relying on key-press inertia.

Why does picking the top probability beat a teacher that rolls the dice?

When we evaluated Gev across 20 tournament matches against eight deathmatch bots, the difference between rolling dice and picking the top choice stood out:

Figure 3: Tournament frags per episode and why picking the highest-probability option strips out exploration noise.

How did imitating a teacher bot produce nearly three times more kills (14.75 ± 0.86 frags per match, with a single-match high of 24 frags)?

The training data came from a bot with random exploration. In sampling, the bot rolls a weighted die across probabilities. In live play, that means it frequently twitches in the wrong direction, hits walls, or hesitates. When Gev samples randomly, it reproduces those noisy twitches and scores 6.15 frags, matching the teacher's 5.00.

When Gev picks its highest-probability option (argmax), it filters out that noise. Across 900,000 examples, it averages over similar states. If the teacher chose the right move 70% of the time and made a mistake 30% of the time, Gev assigns top probability to the 70% consensus.

By executing the highest-confidence move on every frame, Gev moves with decisive momentum. In our blind ablation test (feeding all-black frames), frags dropped to 0.4 and attack rate dropped to 0%, proving Gev only fires when an enemy crosses its sights.

Where to next?

Looking back at the Doom experiment, it was pretty cool to see a compact 270M decision model adapt from reading support tickets to playing a real-time 3D shooter at 35 frames per second, all while keeping its English reading skills intact. Still, it is worth keeping the small-model tradeoff in mind: at 270M parameters, fine-tuning makes Gev a sharp specialist for this specific deathmatch arena, rather than a general-purpose game player that can drop into any video game cold.

Now let's bring everything back to your own day-to-day services. In Part 4, we look at how you can customize Gev for your own production workflows, including zero-shot steering with option descriptions, generating synthetic training data with Gemma 4, and calibrating domain confidence scores.