Same Model, 18 Points Apart: Your Agent Harness Is the Real Product

If you build with AI agents, here's a number worth sitting with: one model, one benchmark, and an 18.4-point spread in results. Nothing changed except the code wrapped around the model.

I'm Justin Pennington, founder of Infraxio, based in Ponte Vedra in St. Johns County, just outside Jacksonville, Florida. We run agents inside our own products every day. This is a builder-focused rewrite of an essay on my site, Stop Shopping for the Best AI Model. Fix What's Around It. More about me and what we build: infraxio.com/justin-pennington.

The data

Stanford's Meta-Harness paper pulled results from the official Terminal-Bench 2.0 leaderboard. Holding the model constant (Claude Opus 4.6) across the 89 tasks:

Two honest caveats. The paper searched for and scored its harness on the same 89 tasks, so treat the top number with some skepticism. And a leaderboard isn't your workload. Still, the direction matches other evidence: Anthropic published research earlier this year on infrastructure noise showing that changing only the resource configuration (same model, same harness, same tasks) moved Terminal-Bench 2.0 results by 6 percentage points.

What "harness" means in practice

The harness is everything except the weights:

• Context: what the agent knows before its first step
• Tools: what it's allowed to call
• State: how it tracks progress across turns
• Verification: how it decides it's actually done

The Stanford harness's headline trick is almost boring. According to the project's write-up, before the agent starts it snapshots its environment (working directory, files, available tools, memory) and puts that in the first prompt. That saves two to five turns of the agent poking around to figure out where it is.

That's not a breakthrough in reasoning. It's a briefing.

Four things to fix before you swap models

1. Front-load the context. Hand the agent the record it needs up front: the customer, the job history, the price sheet, the policy. Every turn spent discovering context is a turn not spent on the task, and a fresh chance to go off track.

2. Scope tools to the job. A short, clearly named tool list beats a long one the model has to guess between. If a task only needs to read a schedule and write a note, don't expose your whole API surface.

3. Make "done" machine-checkable. The quote matches the price sheet. The follow-up got logged. The schedule has no double bookings. If you can't write the check, the agent can't reliably satisfy it either.

4. Keep a small eval set of your own real work. Twenty past jobs with known right answers is enough to start. Run it every time you change the model or the harness. Garry Tan made a related point this week: he posted that eval-driven loops for agent skills are necessary, and that he'd just removed one of his skills because a newer model handled that work better without it. Evals tell you what to add and what to delete.

What this looks like at a small company

At Infraxio we went from about 30 people to seven after adopting AI, and we now deploy a full business hub in under three weeks instead of four to six months. Very little of that came from picking the right model. It came from the unglamorous layer: clean records in IFX Hub, clear permissions on what each agent can touch, and checks that catch a wrong answer before a customer sees it.

Models get swapped every few months. That layer is what you keep.

So try the next model that tops a leaderboard. Just run it through your own eval set first, and if your agents underperform, look at what you're handing them before you blame the weights.

What's the single harness change that moved the needle most for your agents? I'd like to hear it in the comments.

Justin Pennington is the founder of Infraxio, a product studio in Ponte Vedra, Florida. Original essay: penningtonj.com.