The load-bearing wall in your agent is the one you never inspect

A paper came out this week asking whether ML-engineering agents can actually use unlabeled data. It's called AgentSSL. Their agents beat the strongest human-engineered baselines in 8 of 12 settings, which is the part everyone will quote.

The part I can't stop thinking about is the other number. When the final candidate got selected using the agent's own feedback metric — the same internal signal it had been optimizing against — accuracy dropped by up to 47%.

Forty-seven percent. That's not a rounding error or a noisy benchmark. That's the loop confidently picking worse answers than it started with.

I don't think that's a flaw in this one paper. I think it's the shape of almost every agent I've built, and I think most of us have no idea which side of that number we're on.

Generation stopped being the hard part

Two years ago the interesting question was "can the model produce a working solution." Now I can get thirty candidate solutions to a decent-sized refactor problem before my coffee's cold. That part is basically solved, and it got cheap fast.

What didn't get cheap is knowing which of the thirty to ship.

And here's the thing about selection: when you don't have a strong selector, you fall back on whatever signal is already sitting in your loop. The agent already has an opinion about its own output. It's right there, it's free, it doesn't require a test suite. So you wire it in. Score the candidates with the model, take the max, done.

That is the loop. That's the whole design. And it's the one I see in nine out of ten agent repos I read, including several of my own from last year.

A similarity metric wearing a lab coat

When the same model that generated a candidate also scores it, you aren't measuring correctness. You're measuring "how much does this look like the kind of answer I'd produce," which is a similarity metric in a lab coat.

That's fine when the task is stylistic. It is actively dangerous when the task has a right answer, because the self-scorer and the generator share the same blind spots. If the agent consistently misreads one API's calling convention, it will consistently prefer the candidate that misreads it — and it'll do so with a confident little rationale attached.

I've watched this happen in a loop I ran: iterations 4 through 9 all made the score go up, and the actual patch got worse each round. The agent had found a phrase it liked writing. It kept writing it. The number said I was winning.

The AgentSSL result is that failure mode with error bars on it.

What the 47% actually tells you

The drop happened when selection leaned on the agent's own feedback. Which means the rest of the pipeline was fine — the candidates were good, the baseline comparison was real, and the failure lived in one step.

Read that as a diagnostic. If your loop degrades by 47% when you swap one component, that component is doing most of the work. It's the load-bearing wall. And I'd bet it's the piece you have the least test coverage on, because it's the piece that feels like plumbing rather than product.

Your prompt has tests. Your tools have tests. Your retriever has tests. Does your judge have tests? Can you tell me its accuracy? Do you know what it gets wrong, or are you trusting it because it agrees with you a satisfying amount of the time?

What I actually do now

A few things I've moved to, none of them clever:

Score the scorer. Hold out a set of candidates with known-good and known-bad answers. Run your judge over it. Measure it the same way you'd measure any model. If you can't build that set in an afternoon, you don't understand the task well enough to be automating the selection yet.

Decouple the judge. Different model, different prompt, different context window, ideally a different family. Same model scoring itself is the failure mode with a name. I'll take a weaker model as the judge if it hasn't seen the generation context — it doesn't know what it said, so it can't play favorites.

Push work to deterministic checks first. Does the test pass. Does the schema validate. Does the output parse. Does the number match the one in the fixture. Every check you can turn into a boolean is a check the self-eval can't quietly rationalize away. The judge should only see what's left over after the cheap checks are exhausted.

Pairwise, randomized, both directions. Absolute scores from an LLM drift; comparisons are more stable. But run each pair in both orders and throw out the ones that flip. It's a cheap consistency filter and it catches a surprising amount.

Log every selection with the alternatives. Same reason I log tool calls. The bug report you'll get is "it picked something weird yesterday," and if you only stored the winner you have nothing.

Maybe I'm overcorrecting

There's a real case that self-evaluation is fine: tight tasks, crisp verifiers, small candidate sets, loops that run for one or two iterations and stop. If a unit test will reject the bad candidate anyway, the judge's opinion barely matters. I've shipped that.

But notice the size of the correction needed. The moment your agent starts doing multiple rounds — draft, critique, revise, pick — the judge is no longer a filter at the end, it's the gradient. It's the thing steering. Optimizing against your own feedback metric for many rounds is just training, minus the part where training has a held-out set.

So build the held-out set. It's boring and it's the only thing standing between you and a system that gets measurably worse while insisting it's improving.

I'd rather have a mediocre generator and a ruthless selector than the reverse. The generator is a commodity now. The selector is the product, and it's the component I'd bet most of us could not describe accurately if someone asked.