A trailing slash on our site is a 308, a capital letter is a 404, and /. is the homepage answering 200
The URL a reader asks for is not always the URL you published. It arrives with a UTM parameter stapled on, or with a trailing slash from a CMS, or capitalised because somebody typed it off a slide, or with a doubled slas
The URL a reader asks for is not always the URL you published. It arrives with a UTM parameter stapled on, or with a trailing slash from a CMS, or capitalised because somebody typed it off a slide, or with a doubled slash because two bits of string concatenation both added one.
Notifio's site is 38 pages. I have audited what those 38 URLs are before. This time I asked for each of them three different wrong ways, recorded the status code, the redirect target, the <title>, the canonical and every robots meta, and then went after the odd shapes by hand.
The systematic part: 114 requests
For each of the 38 paths: add a trailing slash, upper-case the whole path, add ?utm_source=devto.
| Shape | Answer | Count |
|---|---|---|
/help/ |
308 to /help
|
37 of 38 |
/HELP |
404 | 37 of 38 |
/help?utm_source=devto |
200, canonical points at /help
|
38 of 38 |
The 37s are 38 minus the homepage, where all three variants are the same request. Consistent, which is the boring answer I was hoping for. Two parts of it are worth saying out loud though.
The path is case sensitive and the query string is not part of the identity. /HELP is not a page. /help?utm_source=devto is /help, and it says so: all 38 query variants emit a canonical pointing at the clean URL, with zero exceptions. Those two behaviours together mean the number of distinct URLs that serve our 38 pages is infinite, and the only thing stopping that from being 38 duplicate-content problems is one <link rel="canonical"> per page.
The redirect is cheap and the 404 is not free. A 308 is 15 bytes of body. Our 404 is 13,627 bytes, which is a full page, and the route census that first found it explains why it reads the way it does.
The one-offs, which is where the interesting answers were
//help 308 -> https://notifio.app/help
/help// 308 -> https://notifio.app/help/
/help/?x=1 308 -> https://notifio.app/help?x=1
/HELP/ 308 -> https://notifio.app/HELP
/alerts//kamernet 308 -> https://notifio.app/alerts/kamernet
/%68elp 200 canonical https://notifio.app/help
/%48ELP 404
/. 200 canonical https://notifio.app
/help? 200
/help/extra 404
/index.html 404
/success 200 noindex, nofollow, no canonical
/maintenance 307 -> https://notifio.app/
/alerts/nonexistent 404
Five of those deserve a sentence each.
/help// is a two-hop redirect. It goes to /help/, which is itself a redirect to /help. A doubled trailing slash costs a reader two round trips, and a crawler two hops of its link-equity budget, where the single slash costs one.
/HELP/ redirects into a 404. The slash normalisation runs before anything knows whether the path exists, so an upper-cased URL with a trailing slash gets a 308 to an upper-cased URL without one, and then a 404. Two requests to be told no. There is nothing wrong with this, and it is a good reminder that redirect rules do not know about your routes.
/%68elp is /help. Percent-encoding is decoded before routing, so an encoded lower-case h works and the page comes back with the correct canonical. /%48ELP does not work, because it decodes to /HELP, and we are back to case sensitivity. Encoding is transparent, case is not.
/. answers 200 with the homepage. Title, canonical, full 93,725 bytes of it:
$ curl -s https://notifio.app/. | grep -oE '<title>[^<]*|canonical href="[^"]*'
<title>Notifio – Rental Listing Alert Monitor
canonical href="https://notifio.app"
And its robots meta is index, follow, so that URL is indexable. The canonical is the only thing keeping it from being a second copy of our homepage in a search index. I did not write a rule that allows /. and I cannot find a reason to want one, but the mitigation was already in place before I knew the problem existed, which is the best advertisement for self-referencing canonicals I have.
/success is the one page with no canonical at all. It is the page Stripe sends people to after paying, it is not in the sitemap, and it carries noindex, nofollow, which is right. It is also the page our robots.txt tries to protect with a rule that does not match it:
$ curl -s https://notifio.app/robots.txt
User-Agent: *
Allow: /
Disallow: /api/
Disallow: /success/
Sitemap: https://notifio.app/sitemap.xml
Disallow: /success/ is a literal prefix, and the page is at /success with no slash. That one I had already found and written about, and seeing it again from the other end, by asking the URL instead of reading the rule, is a good argument for measuring from outside.
The 307 on /maintenance is ours: one boolean in the repository decides whether that route renders or sends you home, and with the switch off it sends you home. It is the only 307 in this whole sweep, because it is the only redirect we wrote ourselves rather than inherited.
The two different 404s
Unknown slugs inside our four dynamic route spaces answer differently from unknown paths:
| Request | Size | Title |
|---|---|---|
/index.html |
13,627 | 404: This page could not be found. |
/help/extra |
13,627 | 404: This page could not be found. |
/alerts/nonexistent |
11,895 | Notifio – Rental Listing Alert Monitor |
/compare/nonexistent |
11,899 | Notifio – Rental Listing Alert Monitor |
/for/nonexistent |
11,893 | Notifio – Rental Listing Alert Monitor |
/guides/nonexistent |
11,895 | Notifio – Rental Listing Alert Monitor |
Four route spaces, four 404s within six bytes of each other, all of them wearing the homepage's title, and a completely different 404 for everything else. Both flavours emit two robots metas, noindex and then index, follow, which is the contradiction my earlier route census found and which this sweep now confirms from the wire. The cause is the same in both: there is no not-found.tsx in the app, so the layout's metadata and the framework's default 404 are both rendering.
What I am fixing, and what I am not
Fixing: the missing not-found.tsx. Several of the measurements above are symptoms of that one gap, and a reader who mistypes a URL currently gets a page whose title says it is the homepage.
Fixing: the homepage's own canonical. Our sitemap publishes https://notifio.app/ and the page itself emits https://notifio.app. Both resolve, Google treats them as the same URL, and there is still no good reason for a site's two most authoritative statements about its own front door to disagree by one character.
Not fixing: case sensitivity. A case-insensitive router means every page has 2^n names, and a 404 is a clearer answer than a redirect from a URL nobody published.
Not fixing: /. or the percent-encoded variants. The canonical already handles them, and a rewrite rule for every string that normalises to a real path is a bigger surface than the problem.
If you want the version of this you can run in a minute, it is one line per shape:
for p in /help /help/ /HELP "/help?x=1" //help /help// /. /index.html; do
printf "%-14s %s\n" "$p" "$(curl -s -o /dev/null -w '%{http_code} %{redirect_url}' "https://yoursite$p")"
done
The pages I used are all live: /help, /alerts/kamernet, /compare/rentbird and /guides/rental-scams-netherlands. Notifio is at notifio.app and the app it sells is on the download page.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.