My DNS Check Script Said OK for 41 of 41 Hostnames. 22 of Them Don't Exist.

Two weeks ago I posted a bash script for finding dangling DNS records, the kind of old CNAME that let spammers send mail as marthastewart.msn.com. A reader went through it and left a comment I couldn't argue with: if dig comes back with SERVFAIL, the script falls through to the curl branch, the fetch fails, and the host gets printed as OK.

I promised a fix. Before writing it I wanted to see how bad the gap really was, so I ran the old script against my own domains from the box I do these checks on.

It printed OK for 41 of 41 hostnames. 22 of them don't exist.

Here's what went wrong, the three DNS answers a check script has to tell apart, and the fixed script.

What happened on my box

I pulled every hostname that ever got a certificate on my domains from crt.sh. 13 domains, crt.sh answered for 11 of them (the other two gave a 502 and an HTML error page, twice), 41 unique names.

Old script, default resolver on that machine:

OK        api.earlyhunt.com -> direct
OK        app.auraplusplus.com -> direct
OK        auraplusplus.com -> direct
OK        autodiscover.indiehunt.io -> direct
OK        backend.auraplusplus.com -> direct
...
41 OK

Same script, same list, but with dig pointed at Google's resolver over HTTPS:

GONE      api.earlyhunt.com (no record at all, fine)
GONE      app.auraplusplus.com (no record at all, fine)
GONE      autodiscover.indiehunt.io (no record at all, fine)
...
22 GONE, 19 OK

The reason is boring once you see it. The box's resolver answers every name, real or not, with 198.18.0.1 and status: NOERROR:

$ dig does-not-exist-zq8x7.com A +noall +comments +answer
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51991
does-not-exist-zq8x7.com. 30 IN A 198.18.0.1

That's a sandbox thing in my case, but you get the same behaviour from ISP resolvers that "helpfully" turn NXDOMAIN into a search page, hotel and airport wifi, and some corporate DNS. If your check script trusts whatever resolver the machine has, it is checking the resolver, not your domain.

The three answers a check script has to tell apart

Most scripts, mine included, only look for two states: NOERROR (it exists) and NXDOMAIN (it doesn't). There are really three outcomes, and the third one is two different things.

1. NOERROR or NXDOMAIN. A real answer. You can act on it, as long as the resolver isn't lying (more on that below).

2. Some other status: SERVFAIL, REFUSED. The resolver tried and couldn't give you an answer. To see it, I used dnssec-failed.org, a public test domain with deliberately broken DNSSEC:

$ dig +https @8.8.8.8 dnssec-failed.org A +noall +comments
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 47088
; EDE: 9 (DNSKEY Missing): (No DNSKEY matches DS RRs of dnssec-failed.org)
$ echo $?
0

Two things in there. dig exits 0 on SERVFAIL, so if dig ...; then tells you nothing. And newer resolvers add an Extended DNS Error line that says why it failed. The codes are in RFC 8914, the Extended DNS Errors spec, and they're worth printing when you're debugging a weird host.

The reader's point was that this state shows up in real life when a domain is still registered but its DNS hosting has lapsed, which is often right before it expires. That's exactly the record you want flagged, and the old script called it fine.

3. No answer at all. Timeout, resolver unreachable, VPN dropped mid-run. There's no status line to grep:

$ dig @192.0.2.1 example.com A +noall +comments +time=2 +tries=1
;; communications error to 192.0.2.1#53: timed out
;; no servers could be reached
$ echo $?
9

In the old script status ends up as an empty string. Empty isn't NXDOMAIN, so it goes on to curl, and curl uses the system resolver, which might give a totally different answer than dig did. I pointed the old script at a dead resolver and it printed OK for does-not-exist-zq8x7.com.

The fixed script

Changes from the original:

• A canary first. It looks up a random name that can't exist. If that doesn't come back NXDOMAIN, the script stops instead of printing 41 lies.
• dig's exit code and an empty status both print UNKNOWN.
• Anything that isn't NOERROR or NXDOMAIN prints DNSFAIL on its own line.
• One dig call per host, and the CNAME chain comes from that same answer, so the status and the target can't come from two different lookups.
• curl follows redirects and records the HTTP code. 000 (nothing answered on https or http) is UNKNOWN, a 4xx or 5xx with no fingerprint is CHECK. My first try at the fix didn't have -L, and it flagged www.github.com as unknown because the 301 has an empty body.
• DIG_OPTS so you can pin a resolver without editing the file.

#!/usr/bin/env bash
# usage: ./check-dangling.sh hosts.txt   (one hostname per line)
# needs: dig, curl. Read-only: DNS lookups, one RDAP lookup, one GET per host.
# optional: DIG_OPTS="@8.8.8.8" to pin a validating resolver
FP='There isn.t a GitHub Pages site here|Site not found · GitHub Pages|No such app|NoSuchBucket|DEPLOYMENT_NOT_FOUND|Not Found - Request ID|project not found|Repository not found|Sorry, this shop is currently unavailable'

apex() { echo "$1" | sed 's/\.$//' | awk -F. '{print $(NF-1)"."$NF}'; }  # naive: wrong for .co.uk etc

# canary: a name that can't exist must come back NXDOMAIN, or every answer below is suspect
canary="nx-canary-$RANDOM$RANDOM.com"
cstat=$(dig $DIG_OPTS "$canary" A +noall +comments +time=5 +tries=2 | grep -o 'status: [A-Z]*' | cut -d' ' -f2)
if [ "$cstat" != "NXDOMAIN" ]; then
  echo "STOP: $canary came back '${cstat:-no answer}', not NXDOMAIN. This resolver can't be trusted for this check."
  echo "      Re-run with a public resolver, e.g. [email protected] $0 $1"
  exit 2
fi

while read -r host; do
  [ -z "$host" ] && continue
  out=$(dig $DIG_OPTS "$host" A +noall +comments +answer +time=5 +tries=2); rc=$?
  status=$(echo "$out" | grep -o 'status: [A-Z]*' | cut -d' ' -f2)

  # 1. no answer at all: dig timed out or couldn't reach the resolver
  if [ $rc -ne 0 ] || [ -z "$status" ]; then
    echo "UNKNOWN   $host (no DNS answer, dig exit $rc; check your resolver, then re-run)"; continue
  fi
  # 2. an answer, but not one we can judge: SERVFAIL, REFUSED, ...
  if [ "$status" != "NOERROR" ] && [ "$status" != "NXDOMAIN" ]; then
    echo "DNSFAIL   $host (status $status; look at it by hand, it's not OK and not gone)"; continue
  fi

  last=$(echo "$out" | awk '$4=="CNAME"{t=$5} END{print t}')   # last CNAME in the chain, if any

  if [ -n "$last" ] && [ "$status" = "NXDOMAIN" ]; then
    code=$(curl -sL -o /dev/null -w '%{http_code}' "https://rdap.org/domain/$(apex "$last")")
    if [ "$code" = "404" ]; then
      echo "CRITICAL  $host -> $last  ($(apex "$last") is not registered, anyone can buy it)"
    else
      echo "DANGLING  $host -> $last  (target does not resolve, check if the provider lets others claim it)"
    fi
    continue
  fi

  if [ "$status" = "NXDOMAIN" ]; then echo "GONE      $host (no record at all, fine)"; continue; fi

  # -k on purpose: a dangling host often has the wrong cert, we only want the error page text
  # -L so a plain redirect isn't read as "no page"; 000 = never got an HTTP response
  for scheme in https http; do
    resp=$(curl -skL --max-time 10 -w '\n%{http_code}' "$scheme://$host/")
    code=${resp##*$'\n'}
    [ "$code" != "000" ] && break
  done
  if [ "$code" = "000" ]; then
    echo "UNKNOWN   $host -> ${last:-direct}  (DNS answered, but nothing answered on https or http)"; continue
  fi
  hit=$(echo "$resp" | grep -oE "$FP" | head -n1)
  if [ -n "$hit" ]; then
    echo "CHECK     $host -> ${last:-direct}  (page says: \"$hit\")"
  elif [ "$code" -ge 400 ]; then
    echo "CHECK     $host -> ${last:-direct}  (HTTP $code, nothing matched, look at it)"
  else
    echo "OK        $host -> ${last:-direct}  (HTTP $code)"
  fi
done < "$1"

The same tests, fixed version

Five test hosts: the DNSSEC-broken domain, a name that doesn't exist, example.com, a Vercel app name nobody has deployed, and a CNAME (www.github.com).

With a resolver that tells the truth (DIG_OPTS="+https @8.8.8.8"):

DNSFAIL   dnssec-failed.org (status SERVFAIL; look at it by hand, it's not OK and not gone)
GONE      does-not-exist-zq8x7.com (no record at all, fine)
OK        example.com -> direct  (HTTP 200)
CHECK     zq8x7-none-404.vercel.app -> direct  (page says: "DEPLOYMENT_NOT_FOUND")
OK        www.github.com -> github.com.  (HTTP 200)

On the box's lying resolver, and on a dead one:

STOP: nx-canary-1044327188.com came back 'NOERROR', not NXDOMAIN. This resolver can't be trusted for this check.
STOP: nx-canary-7325282.com came back 'no answer', not NXDOMAIN. This resolver can't be trusted for this check.

Both exit with code 2, so a cron job or CI step fails loudly instead of logging a clean run.

And my own 41 hostnames, on the honest resolver: 22 GONE, 19 OK (all HTTP 200), 0 DNSFAIL, 0 UNKNOWN. Same numbers the old script gave on the honest resolver, which is the point. The old script was right when everything worked and wrong in exactly the cases you run it for.

A note on DIG_OPTS: on a network that intercepts port 53, @1.1.1.1 alone won't help, because the "1.1.1.1" you reach is whatever is doing the intercepting. +https (DNS over HTTPS, in dig 9.18 and later) got around that for me.

What it still doesn't do

• An OK only means the host answered over HTTP. The script uses curl -k on purpose, so it ignores certificate problems. For the hosts that come back OK, I'd run them through a free SSL checker that shows expiry, chain and hostname (it's on Uno Directory, a site I run), because an expired or mismatched cert on an old subdomain is the next thing that bites.
• The apex() function is still naive and gets .co.uk wrong.
• crt.sh only knows names that once had a certificate. A commenter on the old-CNAME post pointed out that an old CNAME which never served HTTPS won't be in CT logs at all, so your zone export is still the real list.
• 14 of my 22 GONE names are cPanel-style ones (cpanel., webmail., webdisk., autodiscover.) that got certificates on a shared host at some point. Old hosting is where forgotten records and forgotten bills both pile up, I wrote about the bills side in keeping side-project bills under $20 a month.

The general lesson I'm taking from this: any check script needs a way to say "I don't know". A check that can only say OK or broken will pick OK when it isn't sure.

What does your DNS or uptime check print when the resolver itself is the thing that's broken? I'm curious if anyone already runs a canary lookup like this in CI.