GPT-6.2 Sol beat GPT-6.1 Sol on the leaderboard. On 5 of my real work orders it tied, and cost 25% more.
OpenAI released GPT-6.2 Sol yesterday at the same price as GPT-6.1 Sol: $2 per million input tokens and $10 per million output. Artificial Analysis puts it at 53 on their Intelligence Index at medium effort, against 46 f
OpenAI released GPT-6.2 Sol yesterday at the same price as GPT-6.1 Sol: $2 per million input tokens and $10 per million output. Artificial Analysis puts it at 53 on their Intelligence Index at medium effort, against 46 for 6.1 Sol, using slightly fewer output tokens. Same price, 7 points higher.
6.1 Sol at medium effort is the default coding worker in my setup, so this morning I spent $11 to find out whether I should switch.
On my orders the two models tied on quality, and 6.2 Sol cost 25% more. I'm not switching yet.
Setup
I plan and review with Claude and hand the implementation to a cheaper model through a small dispatcher I wrote. Each work order runs in its own git worktree inside a macOS sandbox. When the worker says it's done, the dispatcher re-runs the project's test command itself. Then a separate read-only reviewer (Claude Sonnet) reads the diff against the order and returns pass or block. A failed check or a block triggers one automatic retry.
For model comparisons I keep five work orders from October that were hard for the worker I used back then. It needed 3 to 7 attempts on each. Every replay starts from the same base commit as the original. All five are in one Python repo with about 900 tests at those commits.
Today both models got the same five orders at the same moment, through the same Codex CLI build (0.169.0-alpha.11.1), at medium reasoning effort, two jobs per model in parallel. Costs are list prices, including $0.10 per million for cached input.
Results
| GPT-6.2 Sol | GPT-6.1 Sol | |
|---|---|---|
| Passed first time (check green, review pass) | 3/5 | 3/5 |
| Passed in the end, no human input | 5/5 | 5/5 |
| Attempts in total | 7 | 9 |
| Average worker minutes per order | 28.8 | 28.1 |
| Tokens, 5 orders | 27.0M | 19.9M |
| Cost, 5 orders | $5.44 | $4.35 |
Per order:
| Order | GPT-6.2 Sol | GPT-6.1 Sol |
|---|---|---|
| ledger CLI | $0.68 | $0.64 |
| skills-sync CLI | $2.00 | $0.38 |
| wait subcommand | $0.62 | $0.87 |
| hardening follow-ups | $1.25 | $1.32 |
| codex worker adapter | $0.88 | $1.14 |
One order makes the whole difference. Without skills-sync, 6.2 Sol is 13% cheaper ($3.44 against $3.97). With five orders, one outlier decides the headline, so read everything below with that in mind.
The $2 order
The skills-sync order asks for a new script plus tests. 6.2 Sol wrote it, ran the full suite inside the sandbox and saw 5 failures out of 905 tests. None were in its own code. It re-ran four of them in a clean temp directory and they passed. One old assertion still failed. Then it stopped, left its four new files unstaged, and asked me to decide whether it was allowed to commit.
I can defend that call. I also paid for it. The dispatcher's own check on that worktree came back green, found uncommitted files, and sent the order back for an automatic retry. The second attempt read everything again, ran the 13-minute suite again and committed. $1.06 for the first attempt, $0.94 for the second.
6.1 Sol finished the same order in one attempt for $0.38.
Some early feedback on OpenAI's forum says the new models stop too early. This is what that looks like on a bill.
Where 6.2 Sol was better
The codex-adapter order contradicts itself. It says existing tests may only be added to, never changed, but the feature can't go green without changing one old assertion. Both models noticed and said so in their reports.
What they did next was different. 6.2 Sol left the old assertion alone and reported the conflict. When the check failed and the order came back, it changed that one assertion, nothing else, and passed review.
6.1 Sol also loosened the timing parameters of three unrelated watchdog tests to get the suite green. The reviewer blocked that. The order went back at xhigh effort and took four attempts in total, $1.14 against $0.88.
The regression I couldn't reproduce
Artificial Analysis reports 6.2 Sol three points below 6.1 Sol on their Coding Agent Index, measured with the Codex CLI at max effort. They trace the drop to a code Q&A test where the new model leaves out key facts more often. That worried me, because I also send read-only research orders.
So I wrote two. One: list every failure case of a function, with the exact error strings and line numbers. Two: list every exit code, plus every place one particular code is returned. I built the answer keys from the source before running anything, 20 facts and 25 facts.
6.2 Sol got 20/20 and 23/25. 6.1 Sol got 20/20 and 24/25. Two orders is not a benchmark, but at medium effort I saw no sign of the problem.
Speed
Artificial Analysis measures 6.2 Sol as faster per task. I couldn't see it. An order took 25 to 36 minutes on either model, and most of that is the test suite, which needs about 13 minutes per run when four jobs share one Mac mini. The model's thinking time is the small part.
Caveats
Five orders, one repo, one language, medium effort, one day after release. The first-pass rate is understated for both models: at least three of the failed first checks came from timing-sensitive tests under load, not from the model's code. The prompts were written for earlier models and I changed nothing for 6.2.
What I'm doing
6.1 Sol stays my default. For the next week about half of my real orders go to 6.2 Sol, with one added line in its prompt: if the suite has failures that were there before you started, commit anyway and list them in the report. That should remove the $2 case. In a week I'll have 15 to 20 real orders per model and I'll post the numbers.
My advice: before you switch a default model, replay a handful of orders you've already paid for once. Mine cost $11. If you've run the same comparison, I'd like to see your numbers.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.