IndexNow in CI: notify Bing and Yandex on every deploy

IndexNow is a small protocol with one job: tell search engines that a URL was added, changed or deleted, right when it happens. You host a key file, send a POST with a list of URLs, and the engines that take part share the notice with each other.

This post sets it up the way most developers want it: once, inside the deploy pipeline, submitting only the URLs that actually changed. Everything below follows the spec at indexnow.org.

Who receives IndexNow pings

The FAQ at indexnow.org lists these endpoints: the shared api.indexnow.org, Amazon, Bing, Naver, Seznam.cz, Yandex and Yep. A submission to any one of them is shared with all participating engines, so you only need to send it once.

Google is not on that list. For Google you still rely on links, sitemaps and Search Console (more on that at the end). If Bing, Yandex or the other engines send you any traffic, IndexNow is still worth the hour it takes to set up.

Step 1: create and host the key

The key is 8 to 128 characters: letters, numbers and hyphens. A UUID without changes works.

KEY=$(uuidgen | tr 'A-Z' 'a-z')
echo -n "$KEY" > "public/$KEY.txt"
echo "$KEY"

The file must be served at https://yourdomain.com/<key>.txt, as UTF-8, containing only the key. Check it the way a search engine would:

curl -s -o /dev/null -w "%{http_code} %{content_type}\n" \
  "https://www.example.com/$KEY.txt"
# expect: 200 text/plain

Common failures: a framework that rewrites unknown paths to the SPA shell (you get 200 with HTML), a CDN rule that blocks .txt files, or a staging password in front of the whole site.

Each subdomain is a separate host. blog.example.com and shop.example.com each need their own key file.

Step 2: send one URL by hand

Before automating anything, confirm the key works:

curl -i "https://api.indexnow.org/indexnow?url=https%3A%2F%2Fwww.example.com%2Fblog%2Fnew-post%2F&key=$KEY"

The URL must be URL-encoded. Read the status code:

A 200 only means the engine received the URL. It says nothing about crawling or indexing.

Step 3: batch submissions with a small script

For more than one URL, POST JSON. One request takes up to 10,000 URLs, all on the same host.

// scripts/indexnow.mjs
const HOST = process.env.SITE_HOST;          // www.example.com
const KEY = process.env.INDEXNOW_KEY;
const ENDPOINT = "https://api.indexnow.org/indexnow";

export async function submit(urls) {
  const unique = [...new Set(urls)].filter((u) => new URL(u).host === HOST);
  for (let i = 0; i < unique.length; i += 10000) {
    const urlList = unique.slice(i, i + 10000);
    const res = await fetch(ENDPOINT, {
      method: "POST",
      headers: { "Content-Type": "application/json; charset=utf-8" },
      body: JSON.stringify({
        host: HOST,
        key: KEY,
        keyLocation: `https://${HOST}/${KEY}.txt`,
        urlList,
      }),
    });
    console.log(
      new Date().toISOString(),
      res.status,
      `${urlList.length} URLs`,
      res.headers.get("retry-after") ?? ""
    );
    if (res.status === 429) break;          // stop, try on the next deploy
  }
}

if (process.argv[1].endsWith("indexnow.mjs")) {
  const urls = process.argv.slice(2);
  if (urls.length) await submit(urls);
}

The filter on host matters. A stray URL from another domain makes the whole batch fail with 422.

Step 4: work out which URLs changed

This is the part that decides whether IndexNow helps or just adds noise. The FAQ asks you not to resubmit unchanged URLs and to wait at least 5 minutes between submissions of the same URL.

For a static site or a docs site built from Markdown, git already knows what changed. Map content paths to public URLs:

// scripts/changed-urls.mjs
import { execSync } from "node:child_process";

const base = process.env.BASE_SHA || "HEAD~1";
const files = execSync(`git diff --name-status ${base} HEAD -- content/`)
  .toString().trim().split("\n").filter(Boolean);

const urls = files.map((line) => {
  const [, path] = line.split(/\s+/);
  // content/blog/my-post.md -> https://www.example.com/blog/my-post/
  const slug = path.replace(/^content\//, "").replace(/\.mdx?$/, "")
    .replace(/\/index$/, "");
  return `https://${process.env.SITE_HOST}/${slug}/`;
});

console.log(urls.join("\n"));

--name-status includes deleted files (status D). Keep them: IndexNow is also for removals, and the deleted URL should now answer 404 or 410. Redirected URLs can be submitted too.

For a CMS-driven site, hook into the save or publish event instead, and only fire when the body, title, price or availability changed. A typo fix in the footer is not worth a ping.

Step 5: run it on deploy

# .github/workflows/indexnow.yml
name: IndexNow
on:
  workflow_run:
    workflows: ["Deploy"]
    types: [completed]

jobs:
  ping:
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with: { fetch-depth: 2 }
      - uses: actions/setup-node@v7
        with: { node-version: 24 }
      - run: node scripts/indexnow.mjs $(node scripts/changed-urls.mjs)
        env:
          SITE_HOST: www.example.com
          INDEXNOW_KEY: ${{ secrets.INDEXNOW_KEY }}

Run it after the deploy finishes, not before. If an engine fetches the URL a few seconds after your ping and still gets the old version, you've spent the notice for nothing.

Before you write any code: check your platform

The indexnow.org FAQ lists native or plugin support in WordPress, Shopify, Wix, Drupal, Joomla and others, and Cloudflare offers it at the CDN level. If one of those already pings for you, adding your own script doubles every submission. Pick one source.

Measuring whether it works

Bing Webmaster Tools has an IndexNow section that shows the URLs it received from you and any errors. Log every submission with a timestamp, the status code and the URL count, as in the script above, so you can match a deploy to what Bing saw.

Then look at outcomes, not pings. Pick ten URLs you submitted last week and check whether Bing shows the new version. If it doesn't, the cause is usually the page itself: thin content, a noindex, or a canonical pointing elsewhere. IndexNow only shortens the time until a crawler looks.

What about Google?

Google doesn't take part in IndexNow. For your own pages, Google's tools are an XML sitemap with accurate lastmod dates, internal links from pages it already crawls, and the URL Inspection tool for a handful of URLs. Google's recrawl guide covers both and is clear that a request doesn't guarantee inclusion.

A reasonable setup for most sites: IndexNow fires on deploy for the changed URLs, the sitemap regenerates in the same build, and you only open Search Console when a specific page is late.

Written by the IndexerPal team. We build tools for URL and backlink indexing in Google, Bing and Yandex. If you'd rather not run the pipeline yourself, see how IndexerPal indexes websites.