Ahrefs Says 97% of llms.txt Files Get Zero Requests. They Missed the Point.
Ahrefs measured how often AI crawlers fetch llms.txt files and found 97% get zero requests. The study is methodologically sound but the conclusion misses the most important use case: inference-time context injection, not
Ahrefs measured how often AI crawlers fetch llms.txt files and found 97% get zero requests. The study is methodologically sound but the conclusion misses the most important use case: inference-time context injection, not pre-crawl indexing. llms.txt was never primarily a crawler instruction file.
This article was originally published on byfaro.ai/blog/llms-txt-ahrefs-study. Read the full guide there β it includes step-by-step examples and Faro audit tool links.
What you'll learn
This guide covers the full picture on ahrefs says 97% of llms.txt files get zero requests. they missed the point., including:
- Why this matters for AI agent discoverability
- What to check on your own site
- The specific fixes that move your score
Faro audits whether AI agents can find, understand, and act on your business β like GTMetrix but for AI discoverability. Free scan at byfaro.ai.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.