I generated 87 pages from one template. Then I measured how similar they were.
Programmatic pages are easy to ship and hard to audit. You write one template, feed it a list, and get a directory of HTML files. Nothing errors. Every page renders. The nav works, the sitemap validates, and the whole th
Programmatic pages are easy to ship and hard to audit. You write one template, feed it a list, and get a directory of HTML files. Nothing errors. Every page renders. The nav works, the sitemap validates, and the whole thing looks finished.
The part that isn't finished is the part you can't see: how much of each page is actually about that page, and whether the machine-readable files describing your site agree with the files themselves.
I shipped 87 pages that way, then went back and measured them properly. Five of the numbers were not what I expected. Here they are, with the scripts that produced them.
The setup
One Node script, no framework, no bundler. Input is a JSON array of intents:
// intents.json
[
{ "slug": "michael-in-chinese", "family": "transliteration", "subject": "Michael" },
{ "slug": "chinese-names-that-mean-courage", "family": "meaning", "subject": "courage" }
]
Each entry becomes dist/<slug>/index.html. The renderer is a template literal that returns a string; the build writes it out. Across 87 pages: average 1,034 words of body text, roughly 18 KB of HTML per page, and zero <script src> on any content page — plain HTML with inline CSS, which is the one decision here I'd make again without hesitating.
Then I wrote a measurement pass that reads dist/ back and ignores the build's own bookkeeping. That distinction matters, and section 2 is why.
1. Similarity is a number. Measure it before you ship.
"Thin content" and "duplicate content" get argued about as vibes. They're measurable. Shingle the text at n=5 words and take the Jaccard index of every pair:
const shingles = (text, n = 5) => {
const w = text.toLowerCase().split(/\s+/);
const s = new Set();
for (let i = 0; i + n <= w.length; i++) s.add(w.slice(i, i + n).join(' '));
return s;
};
const jaccard = (A, B) => {
const [small, large] = A.size < B.size ? [A, B] : [B, A];
let inter = 0;
for (const x of small) if (large.has(x)) inter++;
return inter / (A.size + B.size - inter);
};
Results over all 3,741 pairs of 87 pages:
- mean Jaccard 0.106
- max 0.664
- min 0.002
- average 1,010 distinct 5-grams per page
A mean of 0.106 sounds great, and it is misleading. The site has three page families, and cross-family pairs sit near zero (the 0.002 minimum is a real pair that shares almost nothing), which drags the average down. The number that matters is within a family. Two adjacent transliteration pages score 0.462 against each other, and 59.6% of page A's 5-grams also appear in page B.
So two pages targeting genuinely different search intents are 60% the same text. That's the actual shape of the problem, and the site-wide average hid it completely.
The maximum pair was worth finding too: 0.664 between two privacy pages I wrote months apart and never diffed. Nobody would have caught that by reading.
What I changed: report similarity per family rather than per site, and fail the build if any intra-family pair exceeds 0.35. Getting under 0.35 forces each page to carry at least one block that is structurally impossible for a sibling to reuse — the actual transliteration table for that specific name, the actual character-by-character breakdown. Boilerplate paragraphs about "why Chinese names are meaningful" will never get you there.
2. Your sitemap is probably generated from the wrong source
Mine was built from intents.json — the list of pages I meant to publish. The site is built from dist/. Those two drift apart the moment a slug is renamed, a page is retired, or a tool page gets added by hand.
Measured drift:
- directories in
dist/: 92 -
<loc>entries insitemap.xml: 87 - directories absent from the sitemap: 5 (one interactive tool page, four asset directories)
None of that is fatal, but it's the wrong shape of truth. A sitemap claiming authority over a set of URLs it never looked at is a document written by hand pretending to be generated.
The fix is to derive it by walking the output directory and assert the counts match:
const urls = fs.readdirSync(DIST)
.filter(d => fs.existsSync(path.join(DIST, d, 'index.html')));
Same bug class as a hand-maintained import list: any file that describes a set must be generated from that set, never alongside it.
3. <lastmod> was recording my build, not my edits
My sitemap stamped lastmod with the build date. Distribution across 87 URLs:
- 53 URLs:
2026-10-20 - 22 URLs:
2026-10-21 - 12 URLs:
2026-10-22 - 1 URL:
2026-10-13
Fifty-three different pages, all claiming to have changed on the same day. Most had not changed at all — the build simply re-ran after I edited a shared component. That's how you teach a crawler to ignore the field.
The fix is a content hash that outlives the process:
const hash = crypto.createHash('sha256')
.update(renderedHtml).digest('hex').slice(0, 12);
const state = fs.existsSync(STATE)
? JSON.parse(fs.readFileSync(STATE, 'utf8')) : {};
state[slug] ??= {};
if (state[slug].hash !== hash) {
state[slug].hash = hash;
state[slug].lastmod = new Date().toISOString().slice(0, 10);
}
fs.writeFileSync(STATE, JSON.stringify(state, null, 2));
Now a rebuild that changes nothing produces a sitemap that changes nothing. That is the entire point of the field, and it requires state outside the build.
4. Title and description lengths are a distribution, not a setting
I checked one page's <title> while building it and it looked fine. Across all 87:
- title length: avg 68, max 80, min 44 — 77 of 87 over 60 characters
- meta description: avg 172, max 216 — 54 of 87 over 160 characters
-
85 of 87 titles end with the same
| Brandsuffix
The length problem is a template problem. One long h1 pattern plus one brand suffix fits the shortest page comfortably and overshoots the longest by twenty characters. There is no single value you can set; you have to check per page and let it fail.
The suffix problem is worse than the length problem. Ten characters of every title spent on a word nobody searches for, repeated across 85 pages, to reinforce a brand the domain name already states. It belongs on the homepage and nowhere else.
5. Don't generate the link graph from the template
Every page linked to the same 21 internal URLs. 87 pages × 21 identical links is very close to a complete graph — every page is equally important, which means none of them is.
Internal links are the one ranking-adjacent signal you fully control, and a uniform template throws it away. Rank candidates by data affinity — same family, shared subject, adjacent concept — then emit the top four to six. Two rules did most of the work:
- never emit a link to a page in the same family that scores above the similarity threshold (you already decided those two are too alike to both deserve ranking)
- never emit more than six; a link block that lists everything is navigation, not a recommendation
The guardrail: six assertions that fail the build
None of this is worth anything as a one-time cleanup. It has to be a verify.js that runs after the renderer and exits non-zero:
const checks = [
['sitemap count matches dist', sitemapUrls === distDirs],
['no orphan directories', orphans.length === 0],
['no bulk lastmod stamping', maxPerDate <= 5],
['titles <= 60 chars', titles.every(t => t.length <= 60)],
['descriptions <= 160 chars', descs.every(d => d.length <= 160)],
['intra-family jaccard <= 0.35', worstPair.jaccard <= 0.35],
];
All six failed on the first run. Three passed by the second. The similarity assertion took real work, because the only way under the threshold is to give every page content that comes from data specific to it.
The general lesson is the one I keep re-learning: programmatic pages don't fail loudly. No exception, no red build step, no 404. You get 87 files that render, a sitemap that validates, and then the slow realisation that the site is one page printed 87 times with a noun swapped. The only way to know which of those you built is to measure the output instead of the input.
I build NameSoul — the site above is what it looks like when a very small team generates its content instead of writing it.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.