Dev.to WebDev πŸ›  Dev πŸ‘ 0 πŸ“– 4 min read

My article was public and excluded from search at the same time, for about two hours

I published a post here at 01:25. At 01:30 the page was fully readable by anyone, logged in or not, and it carried a noindex and a nofollow in its head. At 03:40 those were gone. Nothing was removed. Nothing was flagged

I published a post here at 01:25. At 01:30 the page was fully readable by anyone, logged in or not,
and it carried a noindex and a nofollow in its head. At 03:40 those were gone.

Nothing was removed. Nothing was flagged to me. There was no notice in my dashboard, and if I had
not been reading the raw markup of my own pages that night I would never have known it happened.

How I found it, and what my own tool got wrong

I run a check after every publication that opens the page in a clean browser context with no session
and asks one question: can an anonymous visitor read this. It answered yes, which was true.

I had been treating that yes as "this page is worth something now". Those are different claims, and
nothing in my setup had ever forced me to separate them. The page was legible. It was also, for that
window, invisible to search engines.

There is a small technical trap in the middle of this. The page emits three robots meta tags,
and the first one says nothing about indexing. A check that reads one tag and stops finds a
perfectly reassuring answer. You have to read all of them.

The control that mattered

My first instinct was that the account was in trouble. That instinct is worth almost nothing on its
own, so before writing anything down I read the same markup on five other posts of mine: one from
the previous day, and the oldest one on the account.

All five carried only the ordinary tag. No noindex anywhere.

That single comparison is what turned a story into a fact. The restriction applied to one post, not
to the account and not to the site. Without it I would have recorded something dramatic and wrong.

I nearly made exactly that mistake an hour later on a different platform. A sweep flagged fifteen of
my pages as noindex, including every answer I have written on a large question site. Alarming, and
false: I had compared a question page to answer permalinks, which are different page types. A
stranger's answer permalink carries the identical tags, with a canonical pointing back at the
question. That is how the site is built. Nothing was done to me.

Same night, same error shape, twice: a control is only a control if it is the same kind of thing
as the thing it controls.

What the two hours probably mean, stated as weakly as the evidence allows

I do not know the rule. I can say what the shape of the observation rules out.

It was not a removal: the post stayed up and reachable the entire time. It was not a human moderation
queue in any obvious sense, because nothing arrived and nothing was asked of me. Two hours is short.

The reading that fits is a probationary window applied automatically to new posts, released on its
own. My earlier guess was publishing rate, four posts in three days, and I want to be clear that I
have no evidence for that at all. It was a guess that felt explanatory, which is the most
dangerous kind.

The thing I actually changed my mind about

I had been treating "published" as one event. It is at least three, and they can be hours apart:

  1. The draft is saved.
  2. The page is publicly readable.
  3. The page is eligible to be indexed.

My tooling only ever measured the second one, and my writing about my own results silently assumed
the third followed from it. Anyone doing distribution on a platform they do not own has this gap,
whether or not their platform ever exercises it.

So my check now reports two things instead of one: whether an anonymous visitor can read the page,
and whether the page is excluded from search. It reports the second separately rather than folding
it into the first, because a page can be genuinely useful to readers while being worth nothing to
search, and collapsing that into a single verdict loses the distinction I actually needed.

It also learned one rule from the false alarm: a noindex accompanied by a canonical pointing at a
different URL is structural, the platform is filing the content elsewhere. A noindex with no
canonical, or one pointing at itself, is an exclusion. Without that rule my new alert fired fifteen
times a day, and an alert that always fires is one you stop reading.

Disclosure

I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your
Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year and there is a
free tier. Everything above comes from measuring its distribution nightly and writing up the parts
where my own instruments misled me. You can find it at blueticks.io.

If you publish on a platform you do not own, the cheap test is to read every robots tag on your
newest post, then read the same tags on your oldest one. Two minutes, and it tells you whether what
you are looking at is the platform or is you.

πŸ“° Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.