Chasing milliseconds in Speed Demon 2026: lessons from a solo optimisation sprint

The change I'm proudest of from this contest brought Track A's board memory down from 22.69 MB to 19.69 MB, with similar wall time. The change I was surest about looked faster on my test VM. On the real scoring machine its two runs averaged about 40% slower than the control, and I threw it away the same morning.

I entered Speed Demon 2026, organised by Hackathon Raptors, as team PeppaPig, a team of one. I wrote all four tracks in Rust with no crates, only the standard library. This post is about my method, and the many times my own measurements lied to me.

The contest and the score

There are four tracks. Each is a small, well-defined data job:

• A: Ingest: parse a request log and write per-hour, per-endpoint latency summaries with percentiles.
• B: Index: tokenize a document corpus and answer phrase and boolean queries.
• C: Join: join a quoted CSV of records against a region table and aggregate.
• D: Traverse: build a weighted graph and answer shortest-path queries.

Every output has to be byte-identical to the organisers' Python reference. Correctness is a gate, not a score component. Pass it, and you get a single number, K, built from reference-to-submission ratios for time, memory, source size and dependencies:

• T, wall time (exponent 0.5). This is the big one. The board takes the mean of the middle 3 of 5 runs.
• M, peak cgroup memory (exponent 0.2), taking the highest peak across the five runs. Output written to the tmpfs /out counts against you.
• S, formatted code lines (exponent 0.15). That includes your Dockerfile.
• D, 1 plus direct runtime dependencies (exponent 0.15). With no third-party runtime dependencies, my dependency factor was 1.

The memory ratio is capped at 50; the source and dependency ratios are capped at 1. Wall-time gains are uncapped.

The scoring run gets 6 P-cores and 12 threads on an Intel i5-13500. The input is pre-warmed into the page cache, so the timings focus on warm-cache processing.

S is subtle: while the source-size penalty applies, doubling S costs about 10% of K. Removing counted lines while preserving time and memory gives a predictable gain; a small measured speedup may disappear on the board.

My best board K per track: A 47.32, B 34.36, C 41.77, D 27.59. B's 34.36 came from the live board and is not yet in the repository log; the logged best for the submitted B code is 34.20.

The method

Be honest about who did the typing

A lot of the experimenting was done by AI coding agents (Hark, OpenAI Codex and Kiro) working side by side, with me directing. Each opened its own branches and pull requests, and every one had to beat production on the board like anything else. My job was to pick targets, set the rules of evidence, decide what shipped, and read every report. The agents worked in parallel on profiles, candidates, correctness checks and benchmark preparation. That let one person try far more ideas, and made a strict process vital: many plausible small wins in a night are a great way to fool yourself.

Profile on hardware that looks like the box

My laptop is ARM, so the serious work moved to an x86 cloud VM. I pinned the benchmarks to a fixed CPU set. Builds ran at low priority on separate cores. perf highlighted hot instructions through sampling, and a cgroup harness read memory.peak the way the board does.

One idea per agent, one lock per benchmark

Each agent got one idea and its own directory. Every timed run took a shared file lock, so agents never benchmarked at the same time.

Paired, interleaved, and repeated

A candidate never ran against a number from yesterday. It ran in repeated pairs against the current production build, in alternating order and across input seeds. Track B used profile-guided optimisation (PGO), so there I compared multiple separately trained PGO builds per side.

The board decides

The rule I settled on was to repeat promising candidates on the live board and judge them against a same-track control from the same period, not against my best-ever run. Some obvious losses stopped after one run; noisy candidates needed more repetitions. If the evidence supported keeping a change, it became production. Otherwise it was reverted, and the reason went into a per-track PERF.md log.

Four wins

1. Track A: redesigning the summary for memory

Profiling left me looking for small gains in the parse, but memory became the cheaper target. The old summary copied each hour's latencies into a per-worker scratch buffer. The new one reuses an existing record vector and does an in-place counting sort (American-flag style) to group records by endpoint in output order. Percentile selection happens within each group; the extra full-hour scratch buffer is gone.

On the board, peak memory fell from the control's 22.69 MB to 20.44 and 19.69 MB, with similar wall time. It tied the control on its first run (45.97 vs 45.97, M −2.2 MB) and won the rerun with 46.90, at M 19.69 MB. It shipped.

2. Track B: a collision-free term table, then 16-bit result lists

B spends most of its time tokenizing and looking up terms. The old linear-probe table could require multiple probes and unpredictable branches. The new version builds a hash-and-displace table once, after parsing the queries, for query terms of at most 16 bytes. Those query keys get distinct slots; a lookup uses a 16-bit bucket displacement and an exact two-word key comparison to reject nonmembers. Longer terms still use a HashMap.

On the board, production B had been sitting at about 30.4–30.9. The table scored 32.51, then 30.67 in a normal batch. Parallelising the setup on top of it (query parsing and interning on all 12 threads) gave 32.66 / 32.10, and that shipped.

Then came a tiny one. Storing the answer step's scratch result lists as u16 instead of u32 also trimmed scored S from 761 to 758. On the board it scored 33.62 / 33.80, with T down to about 120 ms from the ~150 ms I started the day with.

3. Track C: smaller buffers, lower M

C reads records.csv through per-thread pread buffers instead of mapping it. Shrinking the read buffer to 32 KiB and the event blocks to 512 entries brought M to 11.0 MB at the same T (22.9 ms). K went from a best of 39.94 to 41.13 / 41.08. Prewarmed input pages can remain charged outside the measured cgroup, but its peak still includes anonymous memory, kernel memory and tmpfs. Cached input is not categorically free.

4. Line trims with identical machine code

The cheapest wins in the contest were deletions, but I only trusted a deletion if I could prove it changed nothing that runs. For Track A, the agents removed selected #[inline(always)] attributes, dropped guards that were provably always true, and folded a few statements together. Then they diffed the disassembly function by function. For the attributes we removed, the hot code was instruction-for-instruction identical. Where removing an attribute did change the code, it stayed.

A's scored S went from 350 to 325. Its board runs averaged 46.96 against production's 45.97–46.90, and the S gain itself is deterministic. B got the same treatment, with scored S falling from 758 to 701: it scored 33.92 and 34.13 and was kept. An earlier D trim reduced S from 407 to 383, but its board K was 23.93 against the previous best of 24.11.

Failures, and what they taught me

VM wins that lost on the board. That opening story was Track A's compact endpoint dictionary using atomic slot publication. On the VM it looked faster. On the board it scored 38.74 and 39.73, with T at 32 and 30 ms. The control scored 45.97 at T 22.4 ms. A deferred-formatting change reduced board memory to about 18.1 MB but went the same way (40.79 / 45.78), and B's "24-byte slots, twice the table" faded over three runs (31.16 → 28.95 → 26.87). Early on, an allocator-tuning change sent A's board memory from about 22 MB to 65.6 MB. The VM is a filter, not a judge.

PGO variation can hide small wins. I compared multiple PGO builds per side before trusting small differences. Single-threaded training does not by itself guarantee identical binaries; that needs checking in the build outputs. (Adding a PGO stage to the Dockerfiles for A, C and D did not improve their logged board K, and those Dockerfile lines count in S.)

Benchmark order bias. One of our scripts biased the run order. For a while, that made a whole batch of comparisons unreliable. Balanced ordering fixed it, and now I don't trust any A/B number unless I know the order was alternated.

Board noise comes in batches. Running the same code again on the board can move K materially. Some batches are just slow: once, B's T jumped from about 150 to 195 ms with no code change. Worse, the tracks don't slow down together: in one batch B was slow while A ran normally, and in another A lost badly while B and D looked fine. "The other tracks were normal" cannot establish that yours was unaffected. So the rule became: repeat promising candidates, and compare against a control from the same track.

What the leaders did better

The evening before the freeze, I compared my numbers with the board leaders track by track. Those comparisons were not preserved in the per-track logs, so I can't substantiate the exact gaps here. My impression was that runtime remained the biggest opportunity; in C, memory also mattered. Board numbers alone couldn't tell me which techniques the leaders used. I treated faster hot loops and better parallel efficiency as hypotheses to test. CPU time divided by thread count is only an idealised work estimate; serial phases and scaling limit how close wall time can get.

Still, I used board results to decide what shipped, and the per-track logs record many of these trials. If you take one thing from this post: your benchmark is a hypothesis about the scoring machine, not a measurement of it.

Code and performance logs (including recorded run IDs, T, M, S, K) are in the team PeppaPig repository, one PERF.md per track.