Not one sentence on 461 of our pages was typed by a person, and the day a filler ingredient started answering the questions

A question page on Munchable's site has three things on it: a question as the heading, a direct answer, and the reason underneath. Here is one, Is mannitol low FODMAP?, and here is the index of all 461 of them.

Nobody wrote the answers. Nobody could: the app that answers the same question on a phone is the product, and a page that restated what the app does in hand-written prose would be a second opinion with no way of noticing it had become wrong. So the pages ask the code, at build time, and print what comes back.

That decision is cheap to describe and has three consequences I did not see coming. This is about those.

The page runs a probe

The site holds an index of ingredients, and the logic lives in a workspace package that both the app and the site compile in (I wrote about that package here). To find out what the product would say about one ingredient, the page assembles a product that contains it and asks:

function probe(tag: string, condition: ConditionId, asMainIngredient: boolean): Probe {
  const ingredientsTags = asMainIngredient ? [tag, ...INERT_FILLER] : [...INERT_FILLER, tag];
  const result = fitCheck({ barcode: '0000000000000', ingredientsTags, /* ... */ }, {
    conditions: [condition],
  });
  const per = result.perCondition[0];
  return { verdict: per?.verdict ?? 'unknown', reason: per?.reasons[0]?.message };
}

It runs twice per page, because a product is not a list of equal parts: the same ingredient near the top of a label and near the bottom are different facts about the food, and the page is only allowed to say that position matters when the two probes actually disagree:

const asMain = probe(entry.tag, condition, true);
const asMinor = probe(entry.tag, condition, false);
const positionMatters = asMain.verdict !== asMinor.verdict;

So the page has the product's own answer, in both readings, before a word of it is rendered. The route is pinned so this can only ever happen at build time:

export const dynamic = 'error';
export const dynamicParams = false;

dynamic = 'error' is the line I would most like other people to copy. A page that is supposed to be static and quietly becomes dynamic is not an outage, it is a slow bill and a cache that stops working, and nobody finds it for a month. This way the build fails instead.

The filler is an input, and inputs get interpreted

INERT_FILLER above is the uninteresting part of the probe product: a few ordinary pantry ingredients, there so the ingredient under test can sit at the top of a list in one probe and further down in the other. Its entire job is to be scenery.

It stopped being scenery. One of the entries was an ingredient that is unremarkable in a pantry and extreme on exactly one axis, which nothing in the engine measured at the time it was added. Then one of the conditions learned to read that axis. From that build onwards every probe for that condition came back with the same answer, because the loudest thing in the probe product was the padding.

The symptom is worth recognising, because it does not look like a bug at first. Nothing crashed, no test failed, and the pages still had answers on them. They were simply all the same answer, and they were about the filler.

The comment above that array is now several times longer than the array itself, and it opens with a line in capitals naming the category of ingredient that must never be added to it, and which part of the engine changed under it. It is the only shouting in the file and it has earned its place.

The general version: if you generate content by calling your own domain logic with a synthetic input, the parts of that input you consider neutral are a standing bet that your logic will never learn to care about them. Your logic exists in order to learn to care about things. Write down what each part of the fixture is for, so that the next person extending the engine reads it in the diff.

There is a milder version of the same surprise in the other direction. One condition generated no pages at all for months, and the reason was not a bug either: what it reads is not something a one-ingredient probe has. A generated page set has a shape, and the shape is decided by what your inputs can actually express, not by what you intended to cover.

A generated page goes missing without making a sound

This is the consequence I would warn hardest about. A hand-written page that breaks gives you a 404 or a build error. A generated page that breaks is simply not generated, and everything downstream agrees with its absence: the sitemap shrinks, the index page lists one fewer link, the build goes green.

So for the page set where there is an explicit promise about which pages exist, there is a test that reads the promise and checks the pages exist and answer something:

it('every promised label word has a page, and the page answers its question', () => {
  for (const word of PROMISED_WORDS) {
    const entry = findEntry(word);
    if (!entry) { failures.push(`${word} is not in the index`); continue; }
    const page = answerPage(slugFor(entry));
    if (!page) failures.push(`${word} has no answer page`);
    else if (page.asMain.verdict === 'unknown') failures.push(`${word} answers unknown`);
  }
  assert.deepEqual(failures, []);
});

Two failure modes sit behind that, and they are different. A typo or a renamed identifier means the word is not a thing the engine recognises at all, so no page is generated. A reorganised data file means the word is still recognised but no longer reaches the data that grades it, so a page is generated and its answer is "cannot say". The first is invisible, the second is worse than invisible: it is a published page that has nothing on it.

Asserting on deepEqual(failures, []) rather than looping with individual assertions is a small thing that pays for itself here. A set of generated pages tends to break in batches, and one failing assertion tells you about one word while the array tells you the shape of what went wrong.

What it costs to not write your own copy

I cannot fix a sentence on those 461 pages. If a line reads awkwardly, the wording lives in a table that owns every page of that shape, and changing it changes all of them, which is the correct behaviour and is nonetheless frustrating about once a month. The counterpart is that when the logic changed last week, 461 pages were correct the moment the deploy finished, and the number of editorial tickets was zero.

The same approach runs the reference page for one condition's whole label vocabulary, where even the group headings and the counts in them are counted rather than typed: hidden gluten on a food label. If a word is added to the data, the page grows a row and the count next to the heading moves on its own.

Pick any page from the index and the thing to notice is the thing you cannot see: no editorial queue behind it, and nowhere for it to drift to.