"Discovered, currently not indexed": a developer's checklist
You shipped a docs page, a changelog, or a new landing URL. A week later, Google Search Console still lists it under Discovered – currently not indexed. The instinct is to open URL Inspection and click Request indexing a
You shipped a docs page, a changelog, or a new landing URL. A week later, Google Search Console still lists it under Discovered – currently not indexed. The instinct is to open URL Inspection and click Request indexing again. Do the technical work first: this status is usually about crawl priority, and it is not the same problem as Crawled – currently not indexed.
What the status actually means
Google has found the URL — from a link, a sitemap, or both — but has not crawled it yet. In URL Inspection the last crawl date is empty, because there is no crawl to date.
Search Console's own explanation is narrower than most posts make it sound: Google typically wanted to crawl the URL, expected that doing so then would overload the site, and rescheduled the crawl. You will also see it on URLs that look low priority: thin or near-duplicate paths, pages with almost no internal links, or a site whose crawl time is spent on filter and archive URLs.
This is not a penalty, and it is not evidence of a broken noindex tag. Google does not promise to crawl or index every URL it knows about. A few discovered-but-not-crawled URLs on a small site can clear on their own. A long list of important URLs is worth a systematic pass.
Crawled – currently not indexed is a different verdict. Google already fetched the page and chose not to keep it, at least for now. That is usually about the page: thin content, a near-copy of something already indexed, or a page that is simply less useful than what already ranks. Requesting a recrawl of an unchanged page does not fix that.
1. Decide whether the URL should be in Google at all
Before you debug anything, answer two questions:
- Should this exact URL appear in search results?
- Is there a better URL for the same content?
Thank-you pages, login screens, staging leftovers, faceted filter URLs, and print or tracking-parameter copies should usually stay out. If the content already has a canonical URL, point the sitemap, the internal links, and the canonical tag there, and stop spending time on the duplicates.
Google's crawl budget guidance is aimed at large or rapidly changing sites, and at sites where many URLs sit in this bucket. The useful idea still applies on a side project: do not offer Google a large inventory of URLs you do not want crawled.
2. Rule out hard technical blockers
A leftover noindex, a robots.txt Disallow, a 404, a 5xx, or a canonical that points elsewhere will beat any indexing request. Check the live URL, including https and the trailing slash your site uses.
# Follow redirects and print response headers
curl -sI -L "https://example.com/docs/install"
# robots.txt at the site root — a leftover "Disallow: /" from staging is a classic
curl -s "https://example.com/robots.txt"
What to look for:
- HTTP status. Google cannot treat a 4xx or 5xx as useful content. If there is a redirect, the final URL is the one that should be in your sitemap and in internal links.
-
X-Robots-Tagand<meta name="robots">. Anoindex(ornone) is a hard stop for crawlers that honour it. Common sources: a WordPress "Discourage search engines" setting, an SEO plugin's per-page flag, or staging headers copied to production. -
robots.txt. ADisallowthat matches this path stops Googlebot from crawling it. Remember the awkward case: to remove a URL from the index you usually need to allow crawling and usenoindex, because Google has to fetch the page to see the tag. - Canonical. If the tag points at a different URL, Google may index that URL instead of this one. Make the canonical, the internal links, and the sitemap agree.
A short Node script for the HTML signals:
// check-index-signals.mjs
const url = process.argv[2];
if (!url) {
console.error("Usage: node check-index-signals.mjs https://example.com/page");
process.exit(1);
}
const res = await fetch(url, { redirect: "follow" });
const html = await res.text();
const pick = (re) => (html.match(re) || [])[0] || null;
console.log({
requested: url,
finalUrl: res.url,
status: res.status,
xRobots: res.headers.get("x-robots-tag"),
title: (html.match(/<title[^>]*>([\s\S]*?)<\/title>/i) || [])[1]
?.replace(/\s+/g, " ")
.trim(),
robotsMeta: pick(/<meta[^>]+name=["']robots["'][^>]*>/i),
canonical: pick(/<link[^>]+rel=["']canonical["'][^>]*>/i),
});
If this pass is clean, the status is probably not a broken tag.
3. Make the sitemap tell one story
A sitemap helps discovery; it does not guarantee indexing. For a URL that is already "Discovered", Google already knows it exists. The sitemap still matters because a messy one wastes crawl time and sends mixed signals. Keep it to canonical, indexable URLs that return HTTP 200. If the live page redirects, list the final URL. If you use a sitemap index, make sure the child file that lists this URL is reachable and submitted.
# Quick look in a single sitemap file (follow child files yourself if this is a sitemap index)
curl -s "https://example.com/sitemap.xml" | grep -F "https://example.com/docs/install"
Also check that robots.txt points at the sitemap you maintain (Sitemap: https://example.com/sitemap.xml).
4. Raise the priority of the pages that matter
If the important URL is technically fine and still sits in "Discovered", treat it as a priority problem:
- Add a descriptive internal link from a page Google already crawls — the homepage, a docs index, or a related article — not only a footer or a sitemap entry.
- Cut or
noindexlarge sets of thin URLs (empty tags, filters, calendars) so they stop competing. For URLs you never want crawled at all,robots.txtis the cheaper signal;noindexstill requires a fetch. - Check Search Console crawl stats for server errors and slow responses. Unhealthy hosts get crawled less.
Then request indexing once for a few key URLs, after a real change. Google's recrawl documentation is blunt: there is a quota, requesting the same URL multiple times does not get it crawled any faster, crawling can take days to weeks, and a request does not guarantee inclusion in results.
A free technical pass, and the rest of the triage
To run those header and HTML checks in one go, the free Not Indexed Checker reads the live page, robots.txt, and sitemap the way a public crawler would. It looks for a non-success status, noindex, a robots.txt rule that blocks Googlebot, a canonical that points elsewhere, or a missing sitemap entry (a large, split sitemap may not be fully opened).
It cannot log into Search Console, and it will not tell you whether Google has already indexed the page. Only Search Console can. A clean result is not a promise that the page will appear in Google. "Discovered – currently not indexed" and "Crawled – currently not indexed" often will not show up there at all, because they are judgements, not broken tags.
If you want the common Search Console reasons — including this one — explained in order, with a short triage and a weekly check, the Not Indexed Fix Guide is a four-page PDF. The introductory price is €4.50 until 1 December 2026 (normally €9), one-time, instant download. It does not replace Search Console, and it does not get a page indexed for you.
I build PIT Toolkit.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.