How I generate llms-full.txt for 1,500 tools at deploy time (so it never goes stale)

A few weeks ago I wrote about serving llms.txt so AI assistants can cite my site accurately (here's that post). The pushback I got was interesting: "nice idea, but who maintains that?" A hand-written list of URLs rots in a month. When a site has hundreds or thousands of pages, any file you maintain by hand will go stale, and a stale llms.txt is worse than none, because it teaches models to cite URLs that 404.

So I stopped maintaining it. A script generates it at deploy time now.

The rule: the registry is the source of truth

My site has a central registry of tools, and every tool page is rendered from it. That is the key precondition: if your CMS or build system knows about every page, an exhaustive plain-text index is a build artifact, not a document.

The generator (scripts/gen-llms-full.js) walks the registry at npm run build and emits llms-full.txt alongside the static output. Deploy regenerates it. Rename a tool, add a category, split a page: the file updates in the same commit. It cannot drift because nobody edits it.

What goes in

The full file for my site is about 120KB of plain text: every tool with its URL, grouped into the 32 categories the site actually uses, with a one-line description per tool. Not marketing copy, just the "what does this page do" line a model needs before deciding to cite it.

A snippet of the shape:

## PDF tools
- Merge PDF files: https://practicalwebtools.com/edit/merge-pdf
- Split PDF by page ranges: https://practicalwebtools.com/edit/split-pdf
- Edit PDF text in the browser: https://practicalwebtools.com/edit/edit-pdf-text

Three decisions worth copying:

1. Flat list per category, not nested prose. Models extract link+description pairs from - description: URL lines with near-zero error. Nested paragraphs bury the URLs.
2. Absolute URLs. Relative paths in a plain-text file get mangled when models splice citations into answers. Absolute URLs survive.
3. One line per tool, no shame about duplication. If a tool belongs in two categories, list it twice with the two descriptions a searcher would use. This is an index, not a database. Redundancy that helps retrieval is free.

llms.txt vs llms-full.txt

The llms.txt convention defines two files: a curated overview (llms.txt) and the exhaustive dump (llms-full.txt). Treat them differently:

• llms.txt is your front door. Curated, honest counts, the 20-40 URLs that answer most questions. I rewrote mine after catching it claiming "500+ tools" with two dead links, a claim a model would happily repeat.
• llms-full.txt is the site map for machines. Every URL, no curation, generated. Mine lists all ~500 tools, the 460+ unit-conversion pairs, and the 12 Creative Studio apps.

Generate the big one. Curate the small one. Never hand-maintain either.

The payoff is citations, not traffic

Six weeks in, the measurable effect is that AI assistants recommend specific tool URLs instead of the homepage. The deep links models cite match real pages because they read them from the index. That is the whole game: every tool page on my site carries its own summary and FAQ structured data, and the llms index makes those pages findable to models in the first place.

If your build system can enumerate your pages, you can ship this in an afternoon. The script is 40 lines. The hard part is the registry, and if you're rendering pages from data anyway, you already have it. Browse the results yourself at practicalwebtools.com, or read the raw llms-full.txt.