The Carmack Pipeline: How DOOM's 1993 Hacks Predict Modern LLM Architecture

"Computers are incredibly fast, accurate, and stupid. Human beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination." — Albert Einstein

John Carmack wrote DOOM in 1993 on hardware that makes a Raspberry Pi look like a supercomputer. No GPU. No FPU on many target machines. 4 MB RAM. And yet DOOM rendered a 3D world at 35 fps, sustained multiplayer over dial-up modem links, and kept all networked machines in perfect lock-step synchrony.

Thirty years later, the engineers building frontier language models at Anthropic, DeepMind, and Meta are solving structurally identical problems — just at different scales. The bottleneck is still the same: your math is too slow for your silicon, so you change your math.

This article traces five direct lines from DOOM's source code to the systems architecture of modern large language models. If you work in systems software or ML infrastructure, the patterns here are your inheritance.

1. The Root Problem: Your Floating-Point is Too Expensive

DOOM's Situation (1993)

The 80386 and 80486SX processors — the mass-market CPUs DOOM was designed for — had no hardware floating-point unit. Trigonometric functions implemented via the C runtime (sin(), cos(), atan2()) invoked the software emulation layer, which was 50–300× slower than native integer arithmetic. For a real-time renderer computing angles, projections, and world-space transformations for every frame, every surface, and every enemy, that cost was catastrophic.

Carmack's solution was to eliminate floating-point from the entire game engine and work entirely in integers.

Modern LLMs: The Same Constraint, Different Exponent

A GPT-4 class model contains hundreds of billions of FP32 parameters. At inference, a single forward pass through the network requires trillions of floating-point multiply-accumulate operations. Even at peak theoretical FLOPS, modern accelerators are memory-bandwidth constrained: the chip can compute faster than it can feed data from HBM memory.

The modern response is quantization: represent weights and activations in lower-precision integer formats.

# Modern INT4 Quantization (bitsandbytes / GPTQ pattern)
import torch

# Standard FP32 linear projection
weight_fp32 = torch.randn(4096, 4096)  # 64 MB

# Quantized INT4 representation
# scale + zero_point per group of 128 weights
weight_int4 = torch.quantize_per_channel(
    weight_fp32, 
    scales=torch.ones(4096) * 0.01,
    zero_points=torch.zeros(4096, dtype=torch.int32),
    axis=0,
    dtype=torch.qint8
)  # ~8 MB — 8× memory savings

The design choice is structurally identical to Carmack's 16.16 fixed-point arithmetic. You sacrifice some precision; you gain massive throughput. The hardware dictates the mathematics.

// DOOM's 16.16 Fixed-Point (m_fixed.h)
#define FRACBITS    16
#define FRACUNIT    (1 << FRACBITS)   // 1.0 = 65536

typedef int fixed_t;

// Multiply: intermediate 64-bit to prevent overflow
fixed_t FixedMul(fixed_t a, fixed_t b) {
    return (fixed_t)(((int64_t)a * b) >> FRACBITS);
}

2. Precomputed Trigonometry: LUTs Then, RoPE Now

DOOM's Binary Angular Measurement (BAM)

Carmack's most elegant decision was rethinking angle representation itself. Instead of storing angles in degrees (continuous floating-point), DOOM stores them as unsigned 32-bit integers where the full circumference maps to 2^32:

// tables.h
typedef unsigned angle_t;

// Cardinal angles as fixed fractions of 2^32
#define ANG45     0x20000000   // 2^32 / 8
#define ANG90     0x40000000   // 2^32 / 4
#define ANG180    0x80000000   // 2^32 / 2
#define ANG270    0xc0000000   // 3 * 2^32 / 4

This is Binary Angular Measurement (BAM). Its killer property: angle wrap-around at 360° is free. When an unsigned 32-bit integer overflows, it wraps modulo 2^32 — which is exactly what you want. No bounds checks. No if (angle >= 360) angle -= 360. No modulo instructions.

The Lookup Table + Phase-Shift Trick

For the actual sin/cos values, DOOM divides the circle into 8,192 discrete fine angles (FINEANGLES = 8192), then maps any angle_t to a table index with a single right shift:

#define FINEANGLES        8192
#define ANGLETOFINESHIFT  19     // 32 - 13 = 19 (maps 2^32 -> 2^13)

extern fixed_t finesine[5 * FINEANGLES / 4];  // 10,240 entries: 360° + 90° extra

// Cosine = sine phase-shifted by 90° = 2048 entries forward
fixed_t* finecosine = &finesine[FINEANGLES / 4];  // from r_main.c

The finecosine pointer simply points 2,048 entries into the finesine array. Looking up finecosine[i] reads finesine[i + 2048] — no extra computation, no second memory allocation. It exploits the trigonometric identity $\cos(\theta) = \sin(\theta + \pi/2)$ as a pure memory trick.

// Hot path: actor movement with zero trig computation
void MoveActor(fixed_t *x, fixed_t *y, angle_t angle, fixed_t speed) {
    int idx = angle >> ANGLETOFINESHIFT;      // Single shift to LUT index
    *x += FixedMul(speed, finecosine[idx]);   // Array read: no sin(), no cos()
    *y += FixedMul(speed, finesine[idx]);
}

RoPE: 2023's Version of the Same Trick

Rotary Position Embeddings (RoPE), introduced in the RoFormer paper and now standard in LLaMA, Mistral, Gemma, and virtually every frontier open-weight model, solves the same problem: how do you embed angular (positional) information into attention dot products without computing transcendental functions in the hot path?

RoPE represents each token position as a rotation matrix parameterized by angles:

$$\theta_i = \text{position} \cdot 10000^{-2i/d}$$

For a model with 128-dimensional attention heads and context windows of 128K tokens, computing sin(θ) and cos(θ) during the forward pass would be prohibitively expensive. The solution: precompute and cache.

# RoPE frequency precomputation (LLaMA-style, transformers library)
def precompute_freqs_cis(dim: int, seq_len: int, theta: float = 10000.0):
    # Precompute inverse frequencies — the "BAM-to-fine-angle" shift equivalent
    inv_freq = 1.0 / (theta ** (torch.arange(0, dim, 2).float() / dim))

    # For every position in the max context window, precompute sin & cos
    t = torch.arange(seq_len)
    freqs = torch.outer(t, inv_freq)

    # Both sin and cos share one contiguous buffer — just like finesine / finecosine
    freqs_cis = torch.polar(torch.ones_like(freqs), freqs)
    return freqs_cis  # Shape: [seq_len, dim/2] — cached, never recomputed

# Forward pass: table lookup, no trig functions in the hot path
def apply_rotary_emb(xq, xk, freqs_cis):
    xq_ = torch.view_as_complex(xq.float().reshape(*xq.shape[:-1], -1, 2))
    xk_ = torch.view_as_complex(xk.float().reshape(*xk.shape[:-1], -1, 2))
    freqs_cis = freqs_cis[:xq_.shape[1]]  # Slice to actual sequence length
    xq_out = torch.view_as_real(xq_ * freqs_cis).flatten(3)
    xk_out = torch.view_as_real(xk_ * freqs_cis).flatten(3)
    return xq_out.type_as(xq), xk_out.type_as(xk)

The structural parallel is exact:

3. Memory Hierarchy: Cache Lines in 1993, SRAM Tiling in 2024

DOOM's Memory Philosophy

Carmack packed finesine and finecosine into a single contiguous array. The table fits in 10,240 × 4 bytes = ~40 KB. This was deliberate: the 486's L1 cache was 8–16 KB, and L2 was absent or slow. Keeping frequently accessed trig data dense and contiguous maximized cache residency.

The same principle applied to the binary angle representation: every angular arithmetic operation on angle_t values is a 32-bit integer add or subtract — the cheapest possible operation on any ISA, fully pipelined, with no memory round-trips.

FlashAttention: Doing the Same Thing on HBM

The core insight of FlashAttention (Dao et al., 2022) is that standard attention algorithms are memory-bandwidth bound, not compute-bound. The bottleneck is reading and writing the $O(N^2)$ attention score matrix to and from HBM memory.

FlashAttention tiles the computation into blocks that fit within GPU SRAM (the equivalent of CPU L1/L2 cache), fusing the softmax, dropout, and matrix multiply into a single kernel:

Standard Attention:
  HBM read (Q, K) → compute QKᵀ → HBM write (S) → HBM read (S) → softmax → HBM write (P) → ...
  Peak memory: O(N²)   |   HBM bandwidth: massive

FlashAttention:
  Tile Q, K, V into SRAM blocks → compute full attention score within SRAM → accumulate
  Peak memory: O(N)   |   HBM bandwidth: minimal

Carmack kept angles as compact 32-bit integers to maximize cache efficiency. Tri Dao restructured attention to maximize SRAM tiling efficiency. Same principle, different silicon, different decade.

4. Bitwise Determinism: Multiplayer DOOM to Distributed RL

DOOM's Lockstep Networking

DOOM's multiplayer (1993) was architecturally remarkable: each machine ran an identical simulation, synchronized only by exchanging player input packets. No game state was sent over the wire — only the discrete angle_t turn commands, button presses, and movement vectors from each player each tic.

This required bitwise deterministic simulation: every machine had to produce the exact same game state from the exact same input sequence. With floating-point arithmetic, this would have been impossible across different FPU implementations and rounding modes. With BAM and 16.16 fixed-point, every arithmetic result was guaranteed identical.

// A "tic command" sent over the network — only 4 bytes
typedef struct {
    char forwardmove;   // Integer forward/backward
    char sidemove;      // Integer strafe
    short angleturn;    // Integer angle delta (BAM-based)
    short consistancy;  // Checksum for desync detection
    byte chatchar;
    byte buttons;
} ticcmd_t;

Distributed RL Rollouts: The Same Requirement

Modern reinforcement learning at scale (Isaac Gym, EnvPool, Brax) runs thousands of parallel environment rollouts across many machines or processes. For reproducibility, debugging, and policy gradient correctness, each environment must produce identically deterministic trajectories given the same seed and action sequence.

Floating-point non-determinism, particularly in GPU operations using non-associative reductions (like torch.sum() over large tensors), violates this. The standard mitigation strategies read directly from the DOOM playbook:

1. Use integer representations for discrete state: Agent positions, grid indices, event counters — keep them as integers.
2. Canonicalize floating-point order of operations: Use torch.use_deterministic_algorithms(True).
3. Avoid non-deterministic CUDA kernels: atomicAdd() on GPUs is non-associative; execution-order-dependent summation produces different results across runs.

Teams building production robotics simulation pipelines (DeepMind, Boston Dynamics, Tesla Autopilot) treat determinism as a first-class correctness requirement — just as id Software did in 1993.

5. The Synthesis: Hardware Sympathy vs. The Bitter Lesson

Richard Sutton's The Bitter Lesson (2019) is one of the most cited essays in AI research. Its core claim:

"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."

Sutton argues that hand-crafted clever tricks — expert-designed feature representations, carefully tuned coordinate systems — are inevitably surpassed by scale and general learning. The lesson is uncomfortable but historically accurate.

Does that contradict everything we've looked at? No. Here is the distinction:

• Sutton's lesson operates on the algorithm-to-problem abstraction level: don't encode domain knowledge into your learning objective; let the model learn it.
• Carmack's lesson (and FlashAttention, and quantization) operates on the hardware-to-algorithm abstraction level: always map your computation to the actual silicon you're running on.

Scaling laws only remain economically viable when the engineers building the systems have deep hardware sympathy. The model that trains in 30 days instead of 90 days — not because it's algorithmically smarter, but because someone understood GPU memory bandwidth and rewrote the attention kernel — that is Carmack engineering applied to 2024 AI.

The bitter lesson tells you what to compute. Hardware sympathy tells you how to actually compute it on real machines that exist.

Conclusion

The five principles Carmack embedded in DOOM's engine — integer arithmetic as a first-class alternative to floating-point, precomputed and cached transcendental functions, phase-shift aliasing for memory efficiency, contiguous cache-friendly data layout, and bitwise determinism for distributed systems — are not 1993 curiosities. They are the structural underpinning of:

• BitNet and INT4 quantization in production LLM inference.
• RoPE in every frontier transformer architecture.
• FlashAttention and GPU-aware kernel fusion.
• Deterministic rollout environments in distributed reinforcement learning.

The hardware changes. The constraints are eternal. When your math is too slow for your silicon, you change your math.

References & Further Reading

• DOOM Source Code (id-Software/DOOM) — Original tables.h, m_fixed.h, r_main.c
• RoFormer: Enhanced Transformer with Rotary Position Embedding — Su et al. (2021)
• FlashAttention: Fast and Memory-Efficient Exact Attention — Dao et al. (2022)
• BitNet b1.58: 1-bit LLMs for All — Ma et al. (2024)
• The Bitter Lesson — Richard S. Sutton
• LLaMA 2: Open Foundation and Fine-Tuned Chat Models — Touvron et al. (2023)
• @TrisH0x2A on X — The original post that sparked this