We pinned our own domain to 127.0.0.1 and our SSRF guard ate two days of voice notes

I build DOS AI (dosai.pro), an AI assistant that answers a small business's customers in WhatsApp, Instagram and Telegram and keeps the leads in a built-in CRM. Customers in Kazakhstan send a lot of voice notes, so the bot transcribes them before it answers. For two days in October it could not hear any of them, and the cause was one line in /etc/hosts.

What changed

We had just moved the platform from a managed cloud to our own server. The hardening script that ran after the move added this line:

127.0.0.1 sb.dosai.pro dosai.pro

The intent was reasonable: the app calls its own storage and auth services hundreds of times a minute, and those calls should not depend on DNS being up. Pin the names to loopback, let the reverse proxy on the box route them by SNI, done.

What broke

The WhatsApp gateway stores every inbound voice note, photo and document in our own object storage and hands the pipeline a link like https://sb.dosai.pro/storage/v1/object/.... Before the pipeline downloads anything from a URL, an SSRF guard resolves the hostname and refuses private, loopback and link-local addresses. That guard exists because the bot also follows links customers paste into the chat, and "fetch whatever the user sent" is exactly how you end up reading your own metadata endpoint.

After the hosts change, sb.dosai.pro resolved to 127.0.0.1 on the server. The guard did what it was written to do: it saw a loopback address and dropped the download. No transcript, no image description, no player in the inbox. The assistant answered "I could not hear the voice note, could you type it?" to every customer who spoke instead of typing.

Over two days: 314 of 339 voice notes, 241 of 248 photos and 92 of 94 documents across 13 projects were lost this way. Nothing crashed. Nothing logged at error level, because a refused unsafe URL is a warning by design. The founder noticed it on his own ad leads.

Why it was not obvious

Three things lined up.

The hosts entry was made by one person as part of infrastructure work. The SSRF guard was written months earlier by another, for a different threat. Neither change was wrong on its own; the combination was.

The failure mode looked like a model problem. "The bot says it cannot hear voice notes" sends you to the transcription code and the prompt first, not to name resolution.

And the guard was correct from its own point of view. A hostname that resolves to loopback is a textbook DNS-rebinding pattern. It had no way to know that this particular name was ours.

The fix

We considered four options.

1. Remove the hosts entry. Simplest, but it brings back the DNS dependency the migration had just removed, and the next person hardening the box will add it again.
2. Allowlist by suffix: anything ending in .dosai.pro is fine. Ownership is not inherited by subdomains; a wildcard record or a forgotten CNAME would open the guard.
3. Skip the IP check for storage URLs only. Fixes this caller, and leaves the same trap for the next internal caller that fetches from our own name.
4. Teach the guard which public names are ours, by exact match, and ask that question after resolving and before the private-IP check.

We went with the fourth. The site host and the storage host come from the same environment variables the app already uses, so there is one source of truth and no second list to forget. The check is an exact hostname comparison, not a suffix. The connection is still pinned to the resolved address, so a request to our name goes to loopback as intended, and the proxy picks the certificate by SNI.

A hostname we do not own that resolves to loopback is still refused. Look-alike names (sb.dosai.pro.evil.com, dosai.pro-something) are still refused.

export function isOwnPublicHost(hostname: string): boolean {
  const h = hostname.toLowerCase();
  if (APP_HOST && h === APP_HOST) return true;
  return isOwnSupabaseHost(h); // exact match against the storage host from env
}

And in the guard, after DNS resolution:

if (isOwnPublicHost(hostname)) return { safe: true, resolvedIp: allIps[0] };
// ...private / loopback / link-local checks follow for everyone else

The regression test mocks the resolver to return 127.0.0.1 for our own host and asserts the URL is accepted, then does the same for a foreign host and asserts it is refused. On the old code two of the five cases fail, which is the point of the test: it had to go red before the fix, or it proves nothing.

What I took from it

A guard that judges by IP address needs to know your public names, because the answer to "where does our name resolve" is not a property of the name. It changes when you move hosting, add a hosts entry, put a proxy in front, or run in a container with its own resolver.

Security checks that fail closed are correct and quiet at the same time. If a refused download is a warning, the warning needs a counter and the counter needs an alert, otherwise you learn about it from a customer.

And when a bot starts saying "I could not hear that" to everyone at once, suspect the plumbing before the model. The model did not change that morning. The server did.