Dev.to AI 🤖 Ai 👁 0 📖 1 min read

Serving HTTP Accept: text/markdown, Content-Signal & /llms-full.txt on Cloudflare Pages (2026 GEO Guide)

Modern AI coding and shopping agents (ClaudeBot, OAI-SearchBot, PerplexityBot, Claude Code) waste up to 85% of their context window parsing inline CSS, SVGs, and layout boilerplate when fetching raw HTML. In February 20

Modern AI coding and shopping agents (ClaudeBot, OAI-SearchBot, PerplexityBot, Claude Code) waste up to 85% of their context window parsing inline CSS, SVGs, and layout boilerplate when fetching raw HTML.

In February 2026, HTTP content negotiation via Accept: text/markdown and the Content-Signal header emerged as the cleanest white-hat Generative Engine Optimization (GEO) pattern for SaaS and Shopify sites:

  • Human browsers (Accept: text/html) receive your normal interactive page;
  • AI agents (Accept: text/markdown) receive 1:1 visible content converted into clean YAML-frontmatter Markdown tables (Vary: Accept), cutting payload size from ~16,000 tokens to ~2,000 tokens with zero SEO cloaking risk.

Testing Your Site Across All 8 GEO Signals

We built a free real-time edge inspector — GeoReady — plus an open-source zero-dependency MCP server (yixuanrobot/geoready-mcp) that audits any URL in 5 seconds across:

  1. AI Crawler Access & robots.txt (OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended + firewall HTTP 403 challenge detection)
  2. /llms.txt & /llms-full.txt (catching broken SPA router index.html fallbacks)
  3. Accept: text/markdown Negotiation & Token Waste %
  4. JSON-LD Entity Graph & ChatGPT Shopping Offers (Organization, Product/SoftwareApplication, FAQPage)
  5. 2026 Content-Signal HTTP Headers (ai-train=yes, search=yes, ai-input=yes)

Minimal Cloudflare Pages _headers Snippet

Drop this into your static output folder's _headers file:

/*
  Content-Signal: ai-train=yes, search=yes, ai-input=yes
  Vary: Accept
  Link: <https://yourdomain.com/llms.txt>; rel="llms-txt", <https://yourdomain.com/llms-full.txt>; rel="llms-full-txt"

Run via Claude Desktop or Cursor (uvx)

{
  "mcpServers": {
    "geoready-auditor": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/yixuanrobot/geoready-mcp",
        "geoready-mcp"
      ]
    }
  }
}

Test your own domain's live score and generate a tailored llms.txt + robots.txt + JSON-LD kit free at https://geoready.aitruelabs.com.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.