Dev.to WebDev 🛠 Dev 👁 0 📖 5 min read

How to fix 'Discovered – currently not indexed' in Google Search Console (a developer's list of causes)

Automated technical SEO checks that catch indexing blocks and deploy regressions — seotto.com In the Pages report of Search Console, under "Why pages aren't indexed", there's a reason that confuses a lot of people: Dis

Automated technical SEO checks that catch indexing blocks and deploy regressions — seotto.com

In the Pages report of Search Console, under "Why pages aren't indexed", there's a reason that confuses a lot of people: Discovered – currently not indexed.

It's easy to read it as "Google looked at these pages and rejected them". That's a different status ("Crawled – currently not indexed"). "Discovered" means something earlier in the pipeline: Google knows the URL exists, from a link or a sitemap, but hasn't crawled it yet. The "last crawled" date for these URLs is usually empty.

So the question isn't "what's wrong with these pages?" but "why hasn't Google got round to fetching them?" The answer is almost always one of a few things, and most of them are fixable from the code side.

Step 1: look at what's in the list

Open the report and export the example URLs. Before changing anything, sort them into groups:

  • Pages you actually want indexed: articles, products, docs pages.
  • Pages you don't care about: filter combinations, internal search results, tag archives, paginated lists, tracking parameter variants, print versions.
  • A mix of both, which is the most common result.

If the list is mostly the second group, the fix isn't to get those indexed. It's to stop generating and linking them, so Google spends its attention on the first group.

Step 2: check which of these causes applies

Cause 1: too many URLs for the site's perceived value

Google decides how much to crawl a site based on how important the site seems and how much its server can handle. A small site that suddenly exposes 200,000 URLs (an import of product variants, auto-generated location pages, an infinite calendar) will see most of them sit in "Discovered" for a long time. Google found them, but it's in no hurry.

Fixes: reduce the number of crawlable low-value URLs. Stop linking parameter combinations, don't list them in sitemaps, canonicalise variants to a main page, and consider whether every auto-generated page deserves to exist at all. Fewer, better URLs get crawled faster.

Cause 2: the pages are only in the sitemap, not linked

A URL that appears in the sitemap but isn't linked from any other page on your site is weak evidence that the page matters. Sitemaps help discovery, but internal links are a much stronger signal of priority.

Check: for a sample of the stuck URLs, search your own site's HTML for links to them. Orphan pages, reachable only through the sitemap, are very common after migrations and on sites with JavaScript-only navigation. Link to important pages from relevant category pages, hubs, or related content, using normal <a href> links in the HTML.

Cause 3: the server slowed down or errored when Google crawled

If Google's crawlers see slow responses, timeouts or 5xx errors, they reduce the crawl rate to avoid overloading the site. Lots of URLs then queue up in "Discovered".

Check the Crawl Stats report (Settings, Crawl stats): look at average response time, the share of responses by status code, and "host status" problems. Then check your own access logs for Googlebot requests that returned 5xx or took several seconds. A response time that rises sharply when crawling increases is a sign your server or database is the bottleneck.

Fixes: cache rendered pages, make sure crawlers get the same cached pages users do, and fix whatever makes uncached pages slow. Return 503 with Retry-After during maintenance, never a slow 200.

Cause 4: the content looks similar to pages already known

When many URLs share a template and most of their text (thin product variants, near-identical location pages), Google may deprioritise crawling more of them. That's not a penalty; it's an expectation that the next page won't add much.

Fixes: merge thin variants into stronger pages, or make each page genuinely different with real content.

Cause 5: it's simply new

A site or section launched recently can have pages in "Discovered" for days or weeks. If the URLs are good, linked and the server is fast, this usually resolves itself.

“Discovered – currently not indexed” is easier to fix when you catch the cause early. seotto.com watches for indexing blocks and deploy regressions that leave new URLs stuck in that state.

Step 3: what not to do

  • Don't spam "Request indexing". It's rate-limited, helps for a handful of important URLs, and does nothing for thousands of them.
  • Don't resubmit the sitemap every day. Google already knows about the URLs; that's what "Discovered" means.
  • Don't add more URLs to fix the problem. "More content" in the form of more auto-generated pages makes cause 1 worse.
  • Don't block the low-value URLs in robots.txt as the only fix if they're already linked everywhere. They'll stay discovered, uncrawled and sometimes still appear in results without a snippet. Remove the links and canonicalise or noindex them so Google can see that.

Step 4: measure whether it's working

Track two numbers monthly: the count in "Discovered – currently not indexed", and, from your logs, the number of distinct URLs verified Googlebot requests per day in the sections you care about. If the second number rises and the first falls after you've cut low-value URLs and improved linking, you're on the right track. Expect weeks, not days.

Separating the URLs you care about from the URLs you've accidentally created is the core of most crawl problems, and it's what we spend much of our time on at seotto.com.

Related reading

How big is your "Discovered – currently not indexed" list, and what turned out to be in it? Filters and tag archives, or real content stuck in the queue?

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.