OpenRouter Translated 16,918 Strings Into 7 Languages for $460 — Here's the Self-Correcting Pipeline Behind It

OpenRouter's website now speaks seven languages besides English: Simplified Chinese, Spanish, Brazilian Portuguese, German, French, Japanese, and Korean. The localization is in beta, and the interesting part is not the language list. It is the pipeline behind it: a daily loop that translates changed strings, checks them in code, has a second model verify the meaning, and retries until the translation is good enough — or flags it for a human.

The whole thing runs on OpenRouter itself, and the team just published a detailed breakdown of how it works, what it costs, and what they threw away along the way. Here is what matters.

The problem: 16,918 strings that change every day

OpenRouter's site holds 16,918 English strings, and some of them change daily. A one-time translation would be out of date within a week. So instead of translating the site once, the team built a loop that runs every day: it finds the strings that changed, translates them, checks them in code, and has a second model verify the meaning against the English.

Translations live in Git next to the code. The team uses the open-source General Translation runtime to wrap strings in React, and commits translations as JSON catalogs that load locally at runtime — no translation service sits in the request path. Each translated entry also stores a fingerprint of the English it came from. When someone edits the English, the fingerprint stops matching and the entry goes back in the queue. That one rule lets the daily run find every stale entry without retranslating the whole catalog.

Five stages, from scan to merge

Every queued string goes through five stages before it reaches the site.

1. Scan. A script scans the frontend and writes every wrapped English string to a source catalog. Each entry records its file and its UI role — button, heading, table header. The role matters because the same English word can need a different translation in a different place. "Queue" on a button is an instruction; translating it as a noun is exactly one of the defects the team later caught.

2. Translate. New and changed strings go to GPT-6.1 Sol in batches of up to 40, using Structured Outputs to hold the reply to a strict JSON schema. Each request carries global and per-language style notes, glossary rules for any product term in the batch, and the UI role of each string. Because every model on OpenRouter takes the same request, trying a different translator means changing only the model slug.

3. Check. Code rejects any reply that breaks an ICU placeholder, a plural branch, a rich-text tag, or leading or trailing whitespace, or that renders a product term in a form the glossary forbids. A rejected batch is resent in smaller pieces, naming the failure, for up to 4 passes by default. The limit is a setting that goes up to 20. A string that still fails after the last pass stays in English and is flagged in the pull request.

4. Verify. A second call to the same model, acting as verifier, reads each translation against the English and lists errors by severity. Any critical or major error — a dropped clause, a reversed meaning — sends the string back to the translator with every objection so far attached. The translator gets up to 3 retries after its first attempt, and the last retry uses more reasoning effort. Counting the check resends, a string can take up to four translation attempts with up to four check passes each, plus a verifier call after each attempt. That puts the worst case at about 20 model calls per string per language.

There is one hard rule: if the verifier is down, the pipeline writes nothing and the string stays in English. A reader who sees English for a day is better off than one who sees an unchecked translation.

5. Review. Each daily batch lands as a pull request. An AI reviewer reads the diff and posts findings, and a person merges the pull request.

What the pipeline actually adds over a bare model

The team measured the pipeline against the same model with a bare two-line prompt — no style notes, glossary, or retries — over 454 cases. The results are a useful reality check on where the value sits.

The check failures falling from 147 to 4 is the glossary and the retries doing their job. The three outside judges — Claude Opus 5.5, Gemini 4 Argon, and DeepSeek V4 Pro — found 109 critical errors in the pipeline's output against 203 in the bare model's. But the overall preference margin, 139 to 117 with 61 ties and 136 identical pairs, is much narrower, and back-translation slightly favored the bare model. The team's own reading: the pipeline's value sits mostly in the specific mistakes it removes, not in making every translation prettier.

What they tried and threw away

The pipeline looks simple now because the evals let the team remove three parts of the first design.

A cheaper translator. They first tested GPT-6 Luna. In a 254-case model comparison it passed 91.7% of cases on the first try, against 99.2% for GPT-6.1 Sol, and was weakest in Korean and Japanese. Its hardest failures were replies the translation framework could not compile at all, and those failures persisted through every retry. The heavier model cost about 3.4 times as much — $1.94 against $0.57 for those 254 cases. They pay the difference because a wrong string that never converges takes more time and effort to fix than getting it right the first time.

A separate judge model. Before the verifier existed, a separate judge scored every translation and anything it flagged was retranslated. In one benchmark run it marked 91% of that run's real defects as uncertain — but it also flagged 11 of the 14 corrected translations and 42 of the 150 clean ones, sending a lot of good work back for retranslation. With the verifier in place, turning the judge off lowered first-pass accuracy from 99.2% to 98.8% and left the final result unchanged. The judge had been sending about 45% of first drafts back. It now stays off by default.

Leaving hard strings for a person. At first, a string the verifier rejected on every retry stayed untranslated until someone fixed it by hand. A full retranslation of the catalog — 16,274 strings into 7 languages — left 97 such strings (0.09%). Now the pipeline ranks the rejected drafts by the verifier's findings, commits the least-wrong one, and flags it in the pull request. The loop always ends in a translation, even when that translation still carries a flagged error, and the flag puts it in front of the person who merges the pull request.

The feedback loop is the real product

The verifier catches most errors before they ship, and readers catch some of the rest. In the Language dialog, a select-to-report feature lets anyone highlight translated text, add a note, and send it. Each report arrives with the language, the page, and the exact text. A model triages it first, picks one of four fixes, and puts the fix up within minutes. An AI reviewer checks the fix, and a person does the final merge.

• Wrong meaning. The string reverts to English the same day, then gets a corrected translation.
• Wrong term. The term goes into the glossary with the accepted form and the forms to reject for that language. Compliance and protocol names such as HIPAA, SOC 2, OAuth, and SAML are pinned in English.
• Wrong tone or punctuation. The rule goes into that language's style notes.
• One bad string. An exact override pins the approved wording, and no later run can change it.

Mistakes the daily review catches after the verifier passed them go into a defects suite. It has 22 cases so far, including "Queue" translated as a noun on a button that needed a verb, "shadow traffic" rendered as road traffic, and a HIPAA setting whose meaning was inverted. Each case forbids the mistake that shipped, so any future model, prompt, or glossary change that brings it back fails the eval.

What it costs

When the team first priced it, a full retranslation of all 16,274 strings into 7 languages — 113,918 translations in total — came to about $430 with translation and verification included. Verification is slightly more than half, about $228, or $0.002 per string per language. The catalog has since grown to 16,918 strings, and a full retranslation of it today costs about $460. The daily loop only sends what changed: a day with 677 changed strings across 7 languages costs roughly $18.

For scale, the same full retranslation at published per-word starting rates, as of November 10, 2026: Smartling Machine Translation would be about $7,200, Smartling AI Translation about $57,300, Gengo Standard about $86,000, and Smartling human translation about $191,000. OpenRouter's pipeline costs about $0.0005 per word — roughly 6% of Smartling's machine translation rate. Those figures cover the translation and verification calls only, not the engineering time to build the pipeline, the calls made by the AI reviewer and the triage model, or the person who merges each daily pull request.

Four parts you can steal

If you are localizing your own product, the team calls out four parts of the design that transfer directly:

1. Fingerprint every translation against its English source, so stale entries re-queue themselves.
2. Let code catch broken placeholders, plural branches, tags, and forbidden terms before any model reviews semantics.
3. Use a verifier that names each error and its severity, and feed those objections back to the translator.
4. Turn every error a human finds into a glossary rule, a style note, an override, or an eval case.

The headline number — $460 to translate a whole site into seven languages — is the least interesting part. The interesting part is the loop: a translator, a verifier, code checks in between, and a feedback path that converts every reported mistake into a rule the system cannot forget. That is what makes the cost sustainable instead of a one-off.

Source: OpenRouter — How OpenRouter Built a Self-Correcting Localization System