3,452 great reviews, 0 AI recommendations: a 5-step check for what ChatGPT says about your product

A Singapore restaurant group has 20 years of reputation and 3,452 genuine reviews, every restaurant rated 4.5★ or higher. We asked three AI engines 49 buyer questions that didn't name it. It was recommended 0 times.

Nothing was wrong with the food. AI just couldn't read the business: ten pages of its site were indexed by Google, and the mobile first screen took 16.8 seconds to load.

The owner had no idea. No analytics tool would have told him, because nobody clicks a link that was never in the answer.

It isn't only restaurants

A US marketing-software company we diagnosed in October was named in 0 of 185 AI answers across 32 buyer questions. The category leader was named in 137. It also appeared in 0 of 130 "best platform" and comparison articles, the kind of pages AI answers lean on when they build a shortlist.

And ranking on Google doesn't cover you. Across 60 buyer questions in our open dataset, only 22 of 307 ChatGPT citations (7.2%) were pages in the same question's Google top 10. Data and method in the earlier post.

So if you build a product, it's worth checking what AI says when a buyer asks for one. Here's the check we run.

The 5-step check

1. Write the questions your buyers actually type, without your brand name.
"Best X for Y in Z", "X vs Y", "is X worth it", "how much does X cost". Mine the wording from sales calls, support tickets and your own search console. 20–30 questions is enough to start.

2. Ask the app, not only the API.
The API is cheap to script, but a signed-in user in the ChatGPT app can get a different answer. Plan, location and whether you're signed in all change it, so ask from the country your buyers are in.

3. Ask each question more than once.
In our data, asking the same question twice repeated only 53 of 222 cited URLs (23.9%). A single answer is one draw, not a ranking.

4. Log three things per answer.

• Are you named, and where in the list?
• Who is named instead?
• Which URLs are cited?

The third one is the most useful. It tells you which of your own pages to fix and which third-party pages need to mention you.

5. Don't quote a share until you have enough answers.
We don't report a mention rate for a brand until it has at least 10 answers behind it. Fewer than that and one lucky answer swings the number.

What usually moves it

From the diagnostics above, the fixes are rarely exotic:

• A site AI can load and read: fast first screen, real text, the facts (what you do, where, for whom, price range) stated plainly on the page.
• Being on the roundups and comparison pages that the answers cite for your category.
• Answering the buyer's actual question on your own page, in the first part of the page.

Then re-ask the same questions a month later and compare.

Limits

The two diagnostics are single businesses, anonymised. The 7.2% and 23.9% figures come from API runs on Singapore questions, not the consumer apps.

If you've run a check like this on your own product, I'd like to hear what surprised you.

Disclosure: I work on a tool in this space. This post has no product link and no offer.