Two strangers broke our API in one afternoon, and every bug was ours

A company-data API for companies, tested by people who had no reason to be gentle.

On the same afternoon, two developers we had never met ran our product against their own standards. One was benchmarking Romanian company lookups for a data marketplace. The other did a cold review of the public site and found four things in about twenty minutes.

Nine of the ten lookups were right. Everything else in this post is about the tenth.

The whole day, in three words.

What they found

None of these were outages. The API was up the whole time, answering in under a second. That is what makes them interesting: the failure was never in the query, it was in everything around it.

The namesake trap

The dangerous one is the domain lookup. Asked about emag.ro, the name search did what a name search does: it ranked companies by name similarity and returned EMAG RO MATRITE PAVAJ SRL and EMAGAZIN ONLINE ROMANESC S.R.L.. Both are real companies. Neither owns the domain. The owner is DANTE INTERNATIONAL SA, a different name entirely, and no amount of fuzzy matching was going to find it.

The fix was to stop treating a hostname as a name. A hostname now goes to a resolver with exactly three ways to say yes, and each one is pinned by identity rather than resemblance:

1. the company's own registry website field equals the host (or a parent domain of it),
2. the site literally prints a CUI, and that CUI resolves to a registry record,
3. the site prints a registration code in the J40/... shape.

When none of those match, the answer is found: false with an empty result list. Name matches still come back, but in a separate candidates field capped by confidence, with a message saying plainly that they are hints, not a match. That distinction is the whole point: an API that guesses and an API that says "I don't know" have completely different failure costs, and only one of them can be trusted in a pipeline.

{
  "host": "altex.ro",
  "found": false,
  "matchedOn": null,
  "total": 0,
  "returned": 0,
  "results": [],
  "reason": "no_identity_on_site",
  "message": "could not identify the firm for altex.ro: altex.ro shows no CUI or registration number we could read; the candidates below are name hints, not a match.",
  "candidates": [{"cui": "2864518", "name": "ALTEX ROMANIA SRL"}]
}

Those candidates are capped by a confidence threshold and never promoted into results or match, so a caller that reads only the answer cannot be fooled by them.

A 200 is not an answer

The blank list was the one I would have shipped. The browser did await response.json().catch(() => ({})) and then treated any 2xx as a result. A response that was empty, or was a proxy's interception page, or was cut mid-body, therefore rendered as a table with no rows and no message — indistinguishable from "this query legitimately matched nothing".

Two changes, and neither is clever:

• an isAnswer() guard: a response is an answer only if it carries a numeric total and a results array. Anything else goes to an error card with a retry button.
• the charge moved out of the happy path. Every search carries a browser-generated id; if the answer never reaches the page, the browser reports that id and the charge is released, once.

The second one had a hole that a reviewer caught, and it is the sort of hole that only shows up if you think about how answers get lost. Releasing the charge was void fetch(...).catch(() => {}) — fire-and-forget. But the failure mode that loses an answer is a severed connection, and a severed connection also eats the refund request. So the code that fixed the money was itself defeated by the same event that caused the problem.

It now uses navigator.sendBeacon first, because the browser hands that to the network stack and it still leaves while the page is unloading, with keepalive fetch as the fallback, and anything that still cannot leave is queued in localStorage and retried on the next load and when the connection returns. Ten-minute age limit on the id, because a refund endpoint that trusts the client is a quota suggestion, not a quota.

What I deliberately did not fix

For a tester on our invite list, the counter is ours, so releasing a charge is honest bookkeeping.

For a developer searching on their own API key, it isn't. Their real cost was consumed upstream on their key, and the dashboard cannot give it back — it only decrements a local counter that means nothing for them. So a key user whose connection drops still pays for a list they never saw.

That would need a server-side delivery ledger: record the upstream request id next to the browser's id, and refund that id, once, inside the same window. It is a bigger change than a bug fix, and pretending otherwise in a comment would have been worse than filing it and saying so in the route itself.

The verification, including the embarrassing part

Each fix landed with the test that fails without it: a hostname that resolves to the owner or says it cannot identify, a non-list 200 that must not render as an empty table, a bad CUI that keeps its own state and its retry box, a refund that survives a dead link.

Green pull requests are not a green merge, though. When the four fixes were combined, CI failed on something no suite had caught: two f prefixes on strings with no placeholders in them, flagged by ruff's F541. The lint step runs before the Python suites in the same job, so it failed, every deploy step behind it was skipped, and the suites never ran at all on the merged branch. Two characters took down the pipeline, and the tests that would have backed it up never got the chance to.

The honest version of "we tested it" is this: 214 test files, 0 failures, run after the lint was fixed — plus the admission that the combination had been untested until someone went looking.

The part that matters for the product

Neither of these people was a customer. One had a benchmark to run, the other had twenty minutes and a habit of trying the boring cases — a mistyped CUI, a domain instead of a company name.

They found five things in an afternoon, and not one of them was in the query engine. All of them were in the seams: what the copy promises, what a 200 means, what happens to the charge when the network dies, what the page does when you type it wrong.

If you run an API, the useful question is not "does it answer correctly". It is "what does it say when it cannot", and "who pays for that answer". Both are easier to answer before a stranger asks them for you.