SERVFAIL: Diagnosing DNSSEC Validation Failures

A SERVFAIL from DNSSEC means a validating resolver couldn't build a trustworthy chain from your domain up to a signed root, and refused to answer rather than return something it couldn't verify. That's the design working as intended — DNSSEC fails closed. The two things worth knowing are which link in that chain actually broke, and how fast you can tell.

Two real incidents, two different root causes

Most SERVFAIL guides walk through a list of hypothetical causes. Two real, documented incidents cover two very different ways this actually breaks in practice.

Slack, September 30, 2021: a correctly signed answer that meant the wrong thing. This was Slack's third attempt at enabling DNSSEC, after two earlier attempts were rolled back for unrelated reasons. The rollout itself worked — signing was verified over a multi-hour validation period, and the DS record went live at the registrar cleanly. Issues appeared one to two hours later, and the cause wasn't a misconfiguration on Slack's side but a bug in Route 53's wildcard handling. Per Slack's own engineering writeup, when a query for a record type the wildcard didn't define hit a DNSSEC-signed Route 53 zone, the signed NSEC response came back with an almost-empty type bitmap — effectively a validly signed proof that no record type existed for that name at all. A client that queried an AAAA record before the A record got that signed "nothing here" answer, and resolvers including Google Public DNS cached it as authoritative. Every subsequent A record query for the same name then failed from that same cached, correctly-signed, wrong answer — a form of unintentional cache poisoning that DNSSEC itself vouched for. Pulling the DS record didn't provide fast relief, since it carried its own 24-hour TTL and resolvers kept trusting the cached signed response until it expired or was manually flushed. Total impact ran close to 24 hours before recovery, affecting under 1% of Slack's active users at any given time, concentrated on smaller resolvers slower to flush caches.

DENIC, May 5, 2026: a signer bug invalidated a fraction of a TLD's signatures. A routine key rollover at DENIC, the registry for .de, went wrong at the infrastructure level. Per DENIC's own final report, a bug in their key-rollover agent generated a separate key pair for each of three hardware security modules instead of one shared pair distributed across all three. All three key pairs carried the same identifiers, including key tag 33834, but only the key published by one HSM was actually valid for the signatures being served. Roughly a third of DNSSEC signatures in the zone validated correctly; the rest produced invalid NSEC3 signatures, and validating resolvers rejected them as bogus. Because NSEC3 covers the entire delegation, this took down lookups for .de domains that had no DNSSEC of their own, not only the ones using it. DENIC's test environment used a single HSM, so the multi-HSM mismatch never surfaced before release. Three internal monitoring tools did flag the invalid signatures, but their notifications weren't processed correctly and no one intervened in time. The disruption lasted about three hours.

The response from the resolver side is worth noting too. Cloudflare's own account of the incident describes deliberately treating .de as unsigned on 1.1.1.1 for the duration — a temporary, acknowledged tradeoff rather than a bug — reasoning that a real answer without validation beat no answer at all for their users trying to reach .de sites during the outage.

The fast path to telling them apart

Four checks, in the order that actually narrows things down fastest:

• Confirm DNSSEC is the cause at all: dig +cd yourdomain.com bypasses validation. If this returns an answer but the same query without +cd returns SERVFAIL, the problem is specifically in the DNSSEC chain, not general resolution.
• Check signature validity first: dig +dnssec yourdomain.com RRSIG shows the expiration window on your signatures. An expired RRSIG is the single most common everyday cause and the cheapest to rule out before chasing anything more subtle.
• Check for a DS and DNSKEY mismatch: query the DS record at the parent and the DNSKEY records at your own nameservers. A DS record with no matching DNSKEY, or the reverse, means a change on one side of a rollover wasn't mirrored on the other side yet.
• Check negative-answer coverage, not only positive records: the Slack pattern, and the one general guides tend to skip — query a record type you know doesn't exist for the name, including at a wildcard, and confirm the signed NSEC or NSEC3 response actually reflects what does and doesn't exist. A signer that synthesizes an overly broad negative answer produces a validly signed proof of absence for records that are, in fact, present.

What continuous DNS monitoring catches here

DNS monitoring that validates the DNSSEC chain daily — DNSKEY, DS, CDNSKEY, and CDS records, alongside 20+ other record types — catches a DS/DNSKEY mismatch after a rollover, or a signature that starts failing, on the next daily check rather than waiting for a user or an incident report to surface it first.

Neither incident above was caused by DNSSEC "not working." DNSSEC did exactly what it's designed to do in both cases: refuse to serve, or vouch only for, an answer that could be verified. The failure, both times, was upstream of the protocol itself — in how a provider's or registry's own signing infrastructure generated the answer DNSSEC then faithfully signed.

Covered in more detail on the WebPixie blog, including the full incident writeups and sources for both.