Your website might be invisible to ChatGPT (and you'd never know)

Last month I noticed something weird: my site ranked fine on Google, but when I asked ChatGPT, Claude and Perplexity about my topic, they never mentioned it. Not once.

So I dug in. Turns out there's a whole layer of "AI visibility" most of us never check.

What I found

1. My robots.txt was blocking AI crawlers I didn't even know existed.
There are 20+ AI bots now: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and more. Some hosting setups and security plugins block them by default. Cloudflare can too.

2. My content was rendered with JavaScript.
Many AI crawlers don't execute JS the way Googlebot does. If your content only appears after hydration, some bots see an empty page.

3. No structured data.
Schema markup helps AI systems understand what your page is: a product, an article, a tool.

How to check your own site (2 minutes)

I ended up building a set of free checkers for this: BotReady. A few worth trying:

• AI Crawler Checker: tests your robots.txt against 21 AI bots
• AI Crawlability Checker: shows what a bot actually sees on your page
• AI SEO Analysis: 40 checks and a score

5 free checks a day, no sign-up.

Quick fixes if you're blocked

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

(Only if you want AI tools to use your content. That's a real choice, not a default.)

Your turn

Run your site through a checker and drop your result in the comments. I'm curious how many of us are accidentally invisible. 👇

Side projects of mine: FormatMath (file converters that run in your browser) and Fee Math (seller fee calculators).