Dev.to WebDev 🛠 Dev 👁 0 📖 3 min read

Update: one page got indexed in 2 days, but my core tool pages are still "never crawled" after 11 weeks

A while ago I wrote about why Google ignored my React SPA tool site and what I changed. I said I'd post an update and be straight about it if things didn't improve. So here it is: it worked for one page and did nothing f

A while ago I wrote about why Google ignored my React SPA tool site and what I changed. I said I'd post an update and be straight about it if things didn't improve. So here it is: it worked for one page and did nothing for the two that matter most.

Some context: PinFetcher is an independent Pinterest downloader suite (not affiliated with Pinterest). I bought the domain as an expired one in August 2026, so it isn't a blank slate. A previous owner ran a WordPress site there, and Google still remembers some of those old URLs.

What happened

  • Media Saver, a page I added in October, was indexed within 2 days of my submitting it in URL Inspection.
  • Pinterest Video Downloader and the image downloader, my two core tools, still show Last crawl: N/A in URL Inspection. That means Googlebot has not fetched them even once. I've requested indexing two or three times a month. Every time: "Indexing requested". Nothing changes.

What I verified on the stuck pages

The same checklist as before:

  • The live test says "URL is available to Google".
  • The rendered HTML has the full content (about 1,700 words), a self-referencing canonical, index, follow, and no noindex.
  • Both pages are in the sitemap and linked from the header on every page.

A quick way to check the raw HTML of any page:

curl -s -o /dev/null -w "%{http_code}\n" https://your-site.com/your-page
curl -s https://your-site.com/your-page | grep -oE '<link rel="canonical"[^>]*>|<meta name="robots"[^>]*>'

What Crawl Stats showed

  • Google is visiting: 655 requests since Aug 5, so it isn't blocked.
  • But 88% of them were "Refresh" (re-checking URLs it already knows) and only 12% were "Discovery". Google rarely finds new URLs on a young domain.
  • 38 requests were 404s on paths I never created (/pl, /he, feed URLs). They're leftovers from the previous owner's site, and Google keeps rechecking them.

The Links report gave me a lesson

I had 8 external links from directories and profiles. All 8 point to the homepage. None points to a tool page. I'd built a handful of listings, and every one sent trust to a single URL. If you want a specific page to be found, link to that page.

What I can't explain

I don't know why Media Saver was fast and the older pages weren't. It's newer, I submitted it, and it has a lot of text, but I can't prove any of that was the reason, so I won't pretend I can.

What I'm doing now

  • Adding deep links to the two tool pages in my directory listings, my YouTube channel and this article.
  • No more Request Indexing on those two pages. It hasn't helped.
  • Checking back in a week and posting the result either way.

Takeaways

  1. A clean live test doesn't guarantee a crawl. It only rules out a bug on your side.
  2. Check the Links report: do your external links point at the pages you want found?
  3. Crawl Stats shows how much of Google's visiting is discovery. If it's mostly refresh, new URLs will wait.
  4. If you bought an expired domain, expect old URLs to show up in your logs for months.

If you've been through a "never crawled" phase on specific pages while others got indexed, what finally changed it for you?

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.