12 times one AI model stood completely alone against the rest
Most coverage of AI disagreement focuses on claims — a fact only one model asserted. Our study found a second, quieter shape: the lone dissent, where exactly one model took a stance every other judged model opposed. Ther
Most coverage of AI disagreement focuses on claims — a fact only one model asserted. Our study found a second, quieter shape: the lone dissent, where exactly one model took a stance every other judged model opposed. There were 12 of them across 24 questions, and they are not checkable the way a claim is. You cannot look up a dissent. You have to weigh it.
We sent the same 24 questions, unchanged, to five mid-tier models — GPT, Claude, Gemini, Grok and Llama. A Council read flagged 52 material divergences. Exactly half involved a lone model: 14 sole-source claims and 12 lone dissents. The dissents, in full:
- SaaS pricing: one model said the $35 tier is the wrong first move entirely, while the other three favoured a tier or grandfathering path.
- Coffee shop second location: one model rejected the rent-ratio test itself — incremental profit after all costs is the only test that matters.
- Indie app pricing switch: one model said an unconditional yes to moving from a $30 one-time unlock to $5/month, while the other three said no or conditional no.
- Oxford vs Tenochtitlan: one model dated Oxford from 1167, giving a roughly 158-year lead, against about 1096 and roughly 230 years from the rest. The verdict — Oxford is older — was unchanged; only the margin moved.
- Monorepo: one model listed "simplified CI/CD" as a monorepo benefit, while the rest treated CI as the main cost — and its own answer elsewhere said pipelines get more complex.
- Monorepo again: the same model recommended Git submodules (and Bazel) to run the monorepo, a stance no other model took.
- Postgres partitioning: one model said partitioning unconditionally improves query speed; the rest said only when queries filter on the partition key.
- Public API default: one model picked GraphQL as the default, 3–1 against, while the rest picked REST.
- TypeScript: one model preferred interface extends over & intersections, while the other three sent intersections to type.
- Remote team communication: one model picked Slack-style chat as the main tool, 3–1 against, while the rest picked async-first.
- Remote team again: the same model argued chat is already effectively asynchronous across time zones — a premise the other answers rejected.
- Family vehicle: one model made seat versatility the deciding factor, while the other three centred sliding doors and step-in height.
Three patterns stand out. First, dissents cluster where there is no ground truth to appeal to: API style, repo structure, team tooling, pricing moves. Second, a dissent can be internally inconsistent — the monorepo answer praised simplified CI/CD and, in the same answer, said pipelines get more complex. A single-model workflow would present either half as settled. Third, some dissents are method challenges, not answer challenges: rejecting the rent-ratio test is not a different number, it is a different way of deciding, and it is exactly the kind of move a synthesis tends to smooth away.
That smoothing is the risk. Across this study, a synthesis that averages a panel deletes the outlier — and on the claim side, the outlier was the highest-value line in its set 8 of 14 times. Dissents deserve the same protection: not automatic belief, but a visible flag that one credible model disagreed, and why.
My practical rule now: if one model adds a fact nobody else mentioned, check the fact first; if one model dissents on a judgment, read its reasons before discounting it — especially when it is challenging the method, not just the answer; and if a dissent contradicts itself, treat that as information about the dissent, not a reason to ignore that dissents exist.
Full study, method and caveats (including that one judge was also a panel member, the rotating judging sets, and a memory-bleed incident we found and disclosed): https://multio.ai/research/model-divergence
Disclosure: I build MultiO (multio.ai) — One subscription. Every top AI model… Auto sends each message to the most cost-efficient model that can handle it. Compare asks up to three models side by side, and every message shows its exact token charge on a receipt. Free trial: 3,000,000 tokens, no card required.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.