Asking ChatGPT 'would people buy this?' is the worst market research you can do

Everyone does it. You paste your landing page into a chatbot and ask: "Would people buy this?"

It says something encouraging, suggests a few tweaks, and you feel validated. That's the problem.

Three reasons one chatbot answer misleads you

1. It's one voice. Your market is students on prepaid data, parents who ask their partner first, and someone who lost money to an online course last year. One assistant answering "as a typical customer" averages all of them away.

2. It tends to agree with you. Chat assistants are tuned to be helpful and agreeable. "Would people buy this?" is a question that invites a yes.

3. Its numbers are too neat. Ask a model to rate purchase intent from 1 to 5 and you get narrow, unrealistic distributions. A 2025 paper on semantic similarity rating (arXiv 2510.08338) found that letting the model answer in its own words, then scoring those words against anchor statements, comes much closer to real survey panels.

What to do instead

Build a population, not a persona, and let more than one model speak.

That's what MarketMirror does. In one Fitloop run (an invented fitness app), 2,999 simulated people were written and interviewed: a third by Claude, a third by Codex, a third by Gemini. Each was asked about money this month, their phone and data plan, who they ask before buying and a time they got scammed. Then they saw the offer.

Here's what happened when the same offer was answered by different models (all simulated):

One model said "maybe". The other said "no" almost every time. If you'd asked just one of them, you'd have walked away with very different confidence about the same launch.

(The Gemini row is Gemini-written people answered by Claude: Gemini's quota ran out mid-run. Those people answered like Claude's, which says the model doing the answering shaped the result far more than the one that wrote the person.)

What this buys you

• A range, not a verdict. When models disagree, the honest answer is "between these two", and you see it.
• The objection that survived every model. All three groups named the same top objection: trust (59.6% overall). That's the one to fix first.
• Free-text reasons per person, grouped by audience, so you know who says no and why.

The limits (read these)

The people are synthetic. Agreement between models is a robustness check on the simulation, not evidence about real buyers. Use it to choose what to test for real, then run that real test small.

Try it

Open source (AGPL-3.0). Works from Claude Code, Codex, Cursor, Gemini or any MCP agent. Ask it in plain words.

👉 https://github.com/DarkWorldCoder/market-mirror (a ⭐ helps other people find it)

Written with help from an AI assistant; every number was checked against the saved run results.