SemiAnalysis GDPx: The Best AI Model Scores Just 12% — What That Actually Means for Developers

Nothing here is sponsored — no affiliate links on this page. Facts below come from the sources listed at the end, checked 2 November 2026. We have not tested GDPx hands-on, and we have not run any of these models on its tasks.

SemiAnalysis published a new benchmark on 2 November 2026, and its headline number is deliberately uncomfortable: the best frontier AI model scores just 12%. That is not a typo, and it is not a claim that AI agents are useless. It is a claim about what today's agents can — and cannot — do when they are asked to do real professional work, end to end, inside a realistic environment, instead of answering a well-scoped question in one shot.

How we put this together

We verified the story against SemiAnalysis's original GDPx publication of 2 November 2026 and the v1.0 leaderboard coverage the same day. We have not run GDPx tasks ourselves, so nothing here is a hands-on finding. Every number below is labelled with where it came from, and the limits section is not a footnote — it is half the story.

Verdict: Do not read 12% as "agents don't work." Read it as "agents are still early at sustained professional work." If you build with agents, GDPx is a useful reality check for where to trust them, where to keep a human in the loop, and which model is strongest for which kind of work — because the leaders are not the same model in every domain.

What GDPx actually is

GDPx covers 11 domains that SemiAnalysis picked not by their share of GDP, but by where it believes frontier intelligence will earn the highest return in the next couple of years: chip design, hardware design, and materials science; cybersecurity, biology, and alignment; finance, legal, and quant trading; recursive self-improvement (RSI); and software engineering. Government, real estate, and insurance are deliberately left out. The argument is blunt: AI capability will not diffuse evenly across the economy, so a benchmark that weights every sector equally would mislead you about where frontier models actually matter.

Each domain was built with a specialised partner — for example, chip design with Aria, Arcverex, and Ergodex; cybersecurity with Arithmetic; biology with Latch Bio; finance with Living Context; legal with Chakra; quant trading with Third Moment; and software engineering with Datacurve. SemiAnalysis says every partner had to pass its agentic quality-assurance process on the first try, with only minor mistakes at worst, and that it dropped prominent partners who failed that bar.

That partnership model is the point. OpenAI's earlier GDPval benchmark, released in September 2025, is still widely cited on model cards, but SemiAnalysis argues it was designed before agents were really a thing: every task was precisely specified and non-interactive, with all the context spoon-fed in the prompt. GDPx tasks, by contrast, put the agent inside a simulated work environment and make it gather context itself.

The finance example makes the difference concrete. One GDPx finance task is seeded by a real transaction — a private-equity fund buying about 62% of a snack company for $880 million plus an earnout — with roughly 4,000 anonymised files across the target's virtual data room and the deal folders of the buyer, the sell-side banker, and the accounting advisor. The agent, working from one party's perspective, has to navigate a roughly 2,000-file deal room to find what it needs, just as a real associate did.

Other domains are built the same way: cyber tasks based on real zero-days with multiple running services, materials-science tasks inside simulated laboratories with battery cyclers and a transmission electron microscope, and hardware-design tasks in live Siemens NX sessions tested on real AWS FPGA instances. Tasks generally allow up to 8 hours, models run in their native harness where one exists — Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon, OpenCode otherwise — and the headline score is pass@1 across three attempts, with all domains weighted equally.

The leaderboard: 12% at the top, 0% in finance and legal

On GDPx v1.0, the headline scores are: Claude Opus 5.5 (Anthropic): 12.0% — best overall; Gemini 4 Carbon (Google): 10.9% — second; GPT-6 Astra (OpenAI): 9.6% — third. OpenAI and Anthropic generally dominate the top of the ranking, SemiAnalysis says, while other labs and open-weight models are meaningfully behind. Gemini 4 Carbon is, in its words, "a real improvement from Google and the only model in the same ballpark as Opus and Astra."

The domain winners are spiky, and that spikiness is itself the story. Opus 5.5 wins quant trading, hardware design, and chip design, and is the best all-around model. Gemini 4 Carbon leads biology and edges out the win in software engineering — the domain most developers will care about first. GPT-6 Astra wins convincingly for recursive self-improvement and is also best for materials science. Finance and legal: basically 0% for every model. SemiAnalysis says its finance and legal tasks are so challenging that basically every model scores exactly 0, and that for legal tasks, no redline produced across 20 rollouts and four models was usable. Not one.

There is an important qualification, and SemiAnalysis states it plainly: Opus 5.5 and GPT-6 Astra have "sandbagged" overall scores because their safety classifiers sometimes refuse tasks in cybersecurity, biology, and alignment. Exclude those three domains and the ranking shifts. The same effect explains a striking sub-result: Muse Wonder Full, with fewer refusals, ranks top on cyber by a meaningful margin, and SemiAnalysis calls it "actually the best available cyber model today." On cost versus performance, the picture it describes matches everyday experience with the models: Luna and DeepSeek fight for the low end, Opus wins the high end, and OpenAI claims the middle.

What this means for developers, honestly

Stop picking a model on a single headline score. GDPx's whole argument is that a general score hides the differences that matter. The best software-engineering model on this benchmark is not the best overall model, and the best cyber model is not either of the top two overall. If you are choosing an agent for a specific job, look at the domain score for that job — and at the cost curve for the volume you actually run.

Agents are good at some professional work and nowhere near done at others. A 12% top score on 8-hour, environment-rich tasks is not a reason to dismiss agents — the tasks are deliberately frontier-hard, and binary grading gives no partial credit for a mostly-right answer. But it is a strong reason not to hand an agent an unsupervised finance or legal workflow today.

Safety classifiers are now a performance variable, not just a policy footnote. If a model refuses cyber or biology tasks that another model completes, that refusal shows up in its score — and in your product. When you evaluate models for a sensitive domain, test refusals explicitly, not just capability.

Benchmarks themselves need the same scepticism as model claims. SemiAnalysis is candid that building a good broad benchmark is hard — it says its own quality process flags nearly 50% of tasks in Terminal Bench 4.0 as broken — and that a benchmark can be gamed by iterating tasks until they pass quality checks. GDPx will be updated every 3 to 6 months, which means today's ranking is a snapshot, not a verdict. Treat GDPx as one strong signal, cross-checked against your own tasks, not as a replacement for testing on your own work.

What GDPx does NOT prove

It does not prove agents are failing. The tasks are designed to be hard for the right reasons, with binary grading and no partial credit. A low score on frontier-hard tasks is compatible with agents being genuinely useful on easier, well-scoped, supervised work.

It does not prove any model is "best" in general. The domain spikes cut against any single-winner story. The honest summary is a table, not a trophy.

It does not settle the finance and legal question. A 0% result on extremely hard tasks tells you those workflows are not agent-ready on this benchmark's terms. It does not tell you that agents cannot help with narrower, supervised pieces of finance or legal work.

We have not verified these results independently. This article reports SemiAnalysis's published benchmark and its own description of its methodology. We have not run GDPx tasks, and no hands-on testing is claimed.

The useful takeaway is narrower, and more actionable, than the headline. Agents have moved from answering questions to doing multi-step work inside realistic environments — and when you measure them that way, the gap between the best model and a finished professional job is still large, uneven across domains, and sensitive to safety refusals. Build with that gap in mind: pick per domain, supervise the hard parts, and test on your own tasks before you trust any leaderboard — including this one.

Sources

SemiAnalysis, "GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains," newsletter.semianalysis.com, 2 November 2026 (original benchmark publication; domain list, methodology, domain winners, safety-classifier qualification, cost/performance summary); Crypto Briefing, "SemiAnalysis unveils GDPx benchmark, and the best AI model scores 12%," 2 November 2026 (v1.0 leaderboard: Opus 5.5 12.0%, Gemini 4 Carbon 10.9%, GPT-6 Astra 9.6%; 3–6 month update cadence); AI Brief / Daily AI Brief, 2 November 2026 (8-hour task limit, native harnesses, pass@1 across three attempts, equal domain weighting). No hands-on testing was performed for this article, and no benchmarks, prices, or availability beyond the sources above are claimed.