Slovak costs 68% more than English in GPT-5. I tokenized 204 languages to find out why
Originally published on devbyte.space. Every time someone in Slovakia sends a prompt to GPT-5, they pay about 68% more than someone in the UK sending the same sentence. Not because of exchange rates. Because of how the
Originally published on devbyte.space.
Every time someone in Slovakia sends a prompt to GPT-5, they pay about 68% more than someone in the UK sending the same sentence. Not because of exchange rates. Because of how the model cuts text into pieces.
I noticed it the boring way: the invoice didn't match the napkin math. English benchmarks and price lists said one thing, the bill for Slovak prompts said another. So I stopped guessing and measured it. Same 1,012 sentences, 204 languages, 35 tokenizers. The whole thing is now the Token Tax Index. This post is the Slovak slice of it.
Short version
- Slovak needs 1.68Γ the tokens of English on the tokenizer behind GPT-4o, GPT-4.1 and GPT-5. On the 2023 GPT-4 tokenizer it was 2.13Γ.
- Across all 35 tokenizers the median Slovak tax is 1.81Γ. Best mainstream model: Llama 4 at 1.56Γ. Worst: Falcon 3 at 2.27Γ.
- Only the European-trained tokenizers get close to parity: EuroLLM 1.35Γ, Teuken 1.41Γ. A Slovak-native tokenizer (SlovakBERT) comes in at 0.77Γ, i.e. cheaper than English. So the gap is a choice, not physics.
- Slovak isn't special. On GPT-5's tokenizer 161 of 204 languages pay at least 1.5Γ, and 42 pay 2Γ or more. Santali pays 13.7Γ.
What "68% more" actually costs
Token pricing is linear, so the ratio is the bill. Three things happen at once:
- Money. A workload that costs β¬1,000/month in English costs β¬1,680 in Slovak on GPT-5, β¬1,800 on Claude Sonnet 4.6, β¬2,070 on Qwen 3. Same prompts, same answers.
- Context. A 128k window holds 128k tokens, not words. In Slovak that's roughly 40% fewer words of your documents, RAG chunks and chat history. The model forgets sooner.
- Latency. Output is generated one token at a time. A Slovak answer with the same content takes 1.6β2Γ as many steps, so it streams visibly slower. Users notice this before anyone notices the invoice.
The part that stings: in raw UTF-8 bytes, Slovak is only 9% longer than English for the same text. The language isn't longer. The tokenizer just doesn't know it.
Why Slovak gets chopped into confetti
A tokenizer is a compression dictionary built from training text. BPE looks at billions of sentences and keeps the chunks that show up often enough to deserve their own entry. English has had the most text on the internet for thirty years, so the dictionary is full of English: "the", "ing", "tion", entire common words. On GPT-5's tokenizer an English token averages 4.9 characters; a Slovak token averages 2.9.
Slovak then makes it worse in three specific ways:
-
Diacritics. Δ Ε‘ ΕΎ Ε₯ Δ ΔΎ Ε Γ΄ Γ€ are two bytes each. If the tokenizer never learned "ΔΎudΓ" as a unit, it may split it into
ΔΎ+ud+Γ, and that first piece can be a single byte of a two-byte character. - Inflection. English has "house, houses". Slovak has dom, domu, domov, domy, domom, domami, dome... Every form is a separate string the tokenizer must have seen often enough to memorize. It hasn't.
- Not enough Slovak in the corpus. This is the real one. Czech, with roughly twice the web footprint, scores a few points better than Slovak on the same tokenizer (1.60Γ vs 1.68Γ on GPT-5). Polish, with four times the speakers, is 1.64Γ. The tax tracks the amount of training text, not the difficulty of the language.
The models that got better, and the ones that didn't
The good news is in the GPT line. OpenAI's 2023 tokenizer (cl100k_base, GPT-4 / 3.5) charged Slovak 2.13Γ. The 2024 one (o200k_base, everything from GPT-4o to GPT-5) charges 1.68Γ. Doubling the vocabulary to 200k entries bought Slovak a 21% discount that nobody announced. Anthropic went the same way: Claude 2 was 2.25Γ, Sonnet 4.6 is 1.80Γ, the 5.5 family is 1.58Γ.
The bad news is in the open-weight models people self-host to save money. Qwen 3 (2.07Γ), Kimi K2 (2.01Γ) and DeepSeek V3 (1.91Γ) are all worse than GPT-5 for Slovak. Excellent models, tokenizers built for Chinese and English. Run them on Slovak and the "cheap" model needs a quarter more compute per sentence than the API you were trying to replace.
| Tokenizer | Slovak | Czech | Polish | Hungarian | German |
|---|---|---|---|---|---|
| EuroLLM | 1.35Γ | 1.34Γ | 1.22Γ | 1.44Γ | 1.22Γ |
| Teuken (OpenGPT-X) | 1.41Γ | 1.06Γ | 1.10Γ | 1.11Γ | 1.14Γ |
| Llama 4 | 1.56Γ | 1.47Γ | 1.40Γ | 1.56Γ | 1.30Γ |
| Claude 5.5 family | 1.58Γ | 1.42Γ | 1.60Γ | 1.56Γ | 1.74Γ |
| Gemma 3 | 1.62Γ | 1.52Γ | 1.51Γ | 1.68Γ | 1.33Γ |
| GPT-4o / 4.1 / 5 | 1.68Γ | 1.60Γ | 1.64Γ | 1.79Γ | 1.31Γ |
| Claude Sonnet 4.6 | 1.80Γ | 1.64Γ | 1.72Γ | 1.78Γ | 1.76Γ |
| Llama 3 | 1.84Γ | 1.50Γ | 1.89Γ | 2.12Γ | 1.58Γ |
| DeepSeek V3 / R1 | 1.91Γ | 1.70Γ | 1.68Γ | 1.99Γ | 1.51Γ |
| Qwen 3 | 2.07Γ | 2.06Γ | 1.76Γ | 2.12Γ | 1.56Γ |
| GPT-4 / 3.5 (2023) | 2.13Γ | 2.10Γ | 1.91Γ | 2.15Γ | 1.59Γ |
Tokens relative to English (1.00Γ). Lower is cheaper. Full table for 204 languages Γ 35 tokenizers on the Token Tax Index.
What I'd actually do about it
You can't change the tokenizer of a hosted model. You can change what you send it.
- Put the tax in the budget before picking a model. Multiply every price-per-million by the ratio for your language. A 20% cheaper model with a 25% worse tokenizer isn't cheaper.
- Keep system prompts and tool schemas in English. They go out on every call and they're the one part of the prompt whose language you control. The user's text can stay Slovak; the 800-token instruction block doesn't have to.
- Size RAG chunks in tokens, not characters, measured with the real tokenizer. A chunker tuned on English docs overflows on Slovak.
- Prefer the newer tokenizer family. Still on a GPT-4-era or Llama 3-era deployment for Slovak? Moving to a 200k-vocab model is a 15β20% cost cut with no quality trade-off.
- For self-hosting, try EuroLLM and Teuken first. They were trained with Central European text on purpose and they're the only ones under 1.5Γ for Slovak.
- Check your own text. I made a small Slovak token calculator that runs in the browser. Paste a real prompt, see the count on each tokenizer, then decide.
How it was measured
Corpus: FLORES-200 devtest, 1,012 sentences from Wikipedia, professionally translated into 204 languages, so every language carries exactly the same information. Each set goes through each tokenizer; the ratio is total tokens for the language Γ· total tokens for English, with a bootstrap 95% CI. Commercial tokenizers (OpenAI, Anthropic) via their official libraries or token-count endpoints; open models via their Hugging Face tokenizers. Everything is in the repo under MIT, the data has a DOI, and anyone can rerun it. Missing a tokenizer? Open an issue.
One caveat I'd rather say out loud: a cheaper tokenizer is not a better model. Teuken compresses Slovak well and still answers worse than GPT-5. The ratio tells you what you pay per sentence, not what you get back. For that I run a separate benchmark, sk-llm-eval, and that's a different post.
I'm Δ½ubomΓr Tchurik, infrastructure & AI engineer in Bratislava. I build open benchmarks for Slovak and other small languages, and mcp-jail, a sandbox for MCP servers. If your language is on the index and the number surprised you, I'd like to hear about it.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.