The Number I'll Cite Next Year Won't Come From the Paper

I lost a week in March to a flat reward curve.

Not a dramatic failure. Loss went down, reward went up, and the model got quietly worse at the thing I actually cared about. I blamed my data. I blamed the learning rate. I blamed the GPU. Then I diffed my config against the appendix and found three differences: a different chat template, a reward model checkpoint two versions behind, and a KL coefficient that had been changed in a footnote I skimmed past.

None of that is fraud. It's just that the setup was never built to be re-run by someone who wasn't in the room.

That week is why I think the headline number in RL post-training papers is about to stop being the number people cite. Not because the papers are dishonest — most aren't — but because the field is starting to notice that a benchmark table scored on a setup nobody re-runs is a claim, not a measurement.

RL makes this worse than SFT, and it's not close

With supervised fine-tuning you have a cheap sanity check: look at the outputs. If the model writes garbage, you know before you ship. The loss can lie but the samples usually don't.

RL post-training inverts that. The reward is the ground truth, and the reward is the single most likely thing to be broken. Reward hacking means the number goes up while the model gets worse — that's not an edge case, that's the default failure mode. And the only reliable detector for it is somebody with no stake in the result running the same pipeline and reporting what they saw.

So when a paper reports +6 on some preference benchmark, what I actually want to know is: who else ran this? Not "did they release code" — code release is necessary and nowhere near sufficient. I've cloned repos where the training script exists, the data pipeline is a TODO, and the reward model is described as "a 7B preference model we trained." Cool. On what. With which annotators. Judged by which snapshot of which API, because that endpoint has been updated twice since you ran it.

The held-out test that actually counts

Here's the eval I trust: an unaffiliated maintainer reimplements the method from the paper plus the repo, hits roughly the same direction of result, and it still beats the baseline they chose themselves.

That's it. That's the whole benchmark. It's slow, it's unglamorous, it doesn't fit in a tweet, and it's the only signal that survives contact with a different cluster, a different data order, and a person who has no incentive to make the numbers work.

I've watched this happen twice now in the last year — a method that looked marginal in the paper turned out to be solid once someone else got it running, and a method that looked strong quietly stopped working when the reward model was swapped for an open one. Both times the second implementation was more informative than the original table. Both times it was a blog post with a repo attached, not a paper.

The prediction

Within about eighteen months, "independently reproduced" becomes a label people cite. Not a vibe, not a footnote — a line item.

Concretely, I expect to see it in three places. Model cards grow a reproduction section next to the eval table, and an empty one reads as a warning. Leaderboards add a column for it, the way they added "verified" for submissions that came with runnable code. And release posts start putting the reproduction link in the first paragraph instead of burying the arXiv link at the bottom, because the reproduction is the thing that convinces a working engineer and the paper is the thing that convinces a reviewer.

The part I'm less sure about is who pays for it. Reproduction is compute, and compute is the thing nobody wants to donate to someone else's result. My bet is that labs start budgeting a small slice of training spend to fund an outside run — a few thousand dollars against a training run that cost six figures is a rounding error, and it buys the one credential that a table can't. If that happens, the label gets real fast.

What would have to change for the label to mean anything

A reproduction is only as good as the artifacts. Right now "we release code" can mean anything from a full pipeline to a loss function and good luck. The minimum viable version:

• Pinned eval harness version, in the repo, not "we used lm-eval-harness."
• Reward model checkpoint with a hash, or an explicit statement that it's proprietary and the numbers are therefore not reproducible.
• The judge prompt, verbatim, plus the judge model snapshot date.
• Seeds and data order, because in RL they matter more than people admit.
• A baseline that the reproducer gets to pick, not the one that flatters the method.

That last one is the important one. A reproduction that only beats the author's weakest baseline isn't a reproduction, it's a remix.

What I do in the meantime

I don't read the table first anymore. I search for a second implementation. If there's only one, I budget a week to reproduce and assume I'll lose it — and I've started writing down where it broke, because that write-up is the artifact I keep wishing already existed when I'm the one searching.

Maybe I'm wrong about the timeline. Maybe labs get good enough at shipping artifacts that the label is redundant, or maybe the field keeps citing tables because citing tables is cheap and reproducing is expensive and nobody gets promoted for confirming someone else's result. The incentive problem is real and I don't have a fix for it.

But I know which number I'll be quoting next year, and it won't be the one in the abstract.

The paper tells you what the authors believe. The reproduction tells you what's true.