Using AI to audit a specialist shop: why we keep unknowns unknown

We run a small pond business with an online shop. One of our current projects is an audit of the koi and pond fish food range: what the product pages already say, what is missing, and what the manufacturers actually confirm.

We are using AI to help with that work. The useful part has been organising information and finding questions worth checking. The difficult part is keeping a plausible answer separate from a verified fact.

This is an account of work in progress, rather than a claim that we have finished or measured a return on it.

Start with the catalogue you actually have

We supplied a WooCommerce CSV export so the audit could start with our existing products and descriptions.

That sounds obvious, but it changes the task. Instead of asking an AI to write a general description of a product, we can ask it to compare a particular row with a particular source and identify the gaps.

An export is still a snapshot. Ours was already a few days old when we supplied it. It helps with reading and searching, but it does not prove that the live page, price, stock status or variants still match.

I would keep those two checks separate:

• Does the proposed information match the manufacturer's evidence?
• Does the record we are about to change still match the live shop?

Getting the first answer right does not make the second unnecessary.

A useful example: protein ingredients

During the food research, a discussion about poultry-free food raised a question that was more useful than another generic product description: could customers filter by the types of protein ingredients declared in a food?

Fish, poultry and insects are all examples of the distinctions people may want to understand. But the filter is only useful if its values are supported.

If an ingredient declaration does not identify an animal source, I do not want the AI to fill in the missing species because it seems likely. Equally, a description that does not mention poultry is not enough evidence to label something poultry-free.

That is where an explicit unknown is helpful.

For example, a proposed audit record could look like this. This is illustrative data, not a claim about a particular food:

{
  "product_id": "example-product",
  "field": "poultry_free",
  "value": null,
  "status": "needs_manufacturer_confirmation",
  "source_url": null,
  "source_text": null,
  "checked_on": null
}

The null is deliberate. It means we have not established the answer. It does not mean yes, and it does not mean no.

A system that turns an empty value into false would lose that distinction. A system that lets the model turn it into true would be worse.

The follow-up task is to find the relevant declaration or ask the manufacturer. Better wording cannot resolve missing evidence.

Keep the evidence beside the claim

For a developer building this sort of workflow, I would make the evidence record at least as important as the proposed copy.

A useful record needs the product or variant identity, the exact claim being checked, the source, the relevant source wording, and when it was checked. It also needs a way to flag a conflict or a question that remains open.

The product identity matters. A statement about one recipe, pellet size or regional version should not quietly become a statement about every item sharing a brand name.

I would also separate evidence states from publishing states:

“Confirmed” should require a person to check the source. Giving the model a field called verified does not make the model its own independent verifier.

Separate research, writing and shop changes

We have separated the catalogue work from the guide writing. That helps keep the questions clear.

The catalogue audit asks what can be stated accurately about a particular product. The guide asks how to explain the choices to a pond keeper. A readable explanation can use experience and context, but it should not quietly introduce an unsupported product specification.

The same checking habits apply to longer pieces of advice. Our pond heating guide is an example of the material this AI-supported content work has helped us develop.

For shop updates, the boundaries need to be narrower still. An improvement to a description or an ingredient attribute should not also rewrite prices, stock, SKUs or variation relationships.

The workflow I would want is a saved before-version, a small proposed change, a check of the fields that are meant to stay intact, and a fresh read of the saved result. Opening the public page matters too: correct source text can still produce a broken table or an awkward mobile layout.

These are checks to build around the writing. A polished draft is only one part of the job.

What has been worth keeping

The audit has given us better questions to investigate and a clearer way to organise the information for a buying guide. We are still researching and completing that work.

I am not claiming that AI has made every description accurate, finished the audit, or improved our search rankings. Those would need their own evidence.

For this kind of specialist shop, my main lesson so far is straightforward: use AI to organise the work, but preserve the difference between what a source says, what we infer, and what we still need to find out.

Disclosure: This article was drafted by an AI assistant from Matt's instructions and the ongoing catalogue and guide work described above.