Bengali costs 6.3x the tokens English does — I counted

The same paragraph costs 278 tokens in Bengali and 44 in English on Llama 3.1's tokenizer — a 6.3x markup for saying the same thing. On Gemma 3, the same paragraph costs 51 tokens in Bengali and 44 in English, a 1.16x markup. The gap is a tokenizer design choice, not physics.

The study that sent me counting

Last week, University of Michigan researchers published "The Language-Energy Divide" at EMNLP 2026 (DOI: 10.18653/2026.emnlp-main.2113). They ran Qwen 3, Gemma 3, and Meta's Llama 3.1 on reading-comprehension questions from the Belebele dataset in 122 languages, measuring energy with Michigan's ML.Energy framework on the university's Great Lakes cluster. The headline: models use 1.4 to 180 times more energy on non-English text than on equivalent English. Hindi costs 7.9x the energy of English. Bengali costs 15.3x.

The mechanism they name is token count: non-English text shatters into more tokens, and every token costs compute. That claim is checkable on a laptop, so I checked it — with the actual tokenizers from all three models in the study, plus GPT-4o's for comparison.

What I ran

I wrote one three-sentence paragraph in English and translated it into eight other languages myself — French, German, Spanish, Chinese, Japanese, Arabic, Hindi, Bengali. Then I encoded every version with four real tokenizers: Qwen3-8B (official tokenizer.json), Gemma 3 4B (unsloth mirror of the official tokenizer), Llama 3.1 8B (same), and GPT-4o's o200k_base via tiktoken. No model inference, no API calls — pure tokenization, which is exactly the input-side half of the study's energy story.

from tokenizers import Tokenizer
import tiktoken

toks = {
    "Qwen3":    Tokenizer.from_file("tokenizers/qwen3.json"),
    "Gemma 3":  Tokenizer.from_file("tokenizers/gemma3.json"),
    "Llama 3.1": Tokenizer.from_file("tokenizers/llama31.json"),
}
enc = tiktoken.get_encoding("o200k_base")  # GPT-4o

for name, tk in toks.items():
    for lang, text in PARA.items():
        n = len(tk.encode(text).ids)
        print(name, lang, n, "tokens,", len(text.encode()), "bytes")

The scoreboard

Tokens for the same paragraph (English = 43–44 tokens everywhere). Multiplier vs English in parentheses:

Language   Qwen3        Gemma 3      Llama 3.1    GPT-4o
English    43 (1.00x)   44 (1.00x)   44 (1.00x)   43 (1.00x)
French     69 (1.60x)   54 (1.23x)   72 (1.64x)   51 (1.19x)
German     74 (1.72x)   55 (1.25x)   78 (1.77x)   56 (1.30x)
Spanish    64 (1.49x)   55 (1.25x)   66 (1.50x)   54 (1.26x)
Chinese    40 (0.93x)   47 (1.07x)   59 (1.34x)   50 (1.16x)
Japanese   69 (1.60x)   55 (1.25x)   70 (1.59x)   74 (1.72x)
Arabic     61 (1.42x)   62 (1.41x)   75 (1.70x)   56 (1.30x)
Hindi      203 (4.72x)  54 (1.23x)   113 (2.57x)  65 (1.51x)
Bengali    244 (5.67x)  51 (1.16x)   278 (6.32x)  70 (1.63x)

I re-ran the check on a single short sentence ("The library opens in eighteen months.") to make sure the paragraph result wasn't a fluke. Same shape: Bengali 39 tokens on Qwen3 vs 7 in English (5.57x), 45 vs 8 on Llama 3.1 (5.62x), 14 vs 8 on Gemma 3 (1.75x).

Three things the numbers actually say

1. Bengali and Hindi are the tax bracket, and the tax is uneven. Bengali is the most expensive language on three of the four tokenizers, Hindi on Qwen3 is nearly as bad at 4.72x. These are languages spoken by hundreds of millions of people, and the tokenizer treats their scripts as if each character were a rare event. On Llama 3.1, Bengali runs at 0.463 tokens per byte — the tokenizer is spending almost half a token per byte of text, which is close to byte-level fallback. English runs at 0.189.

2. Gemma 3 proves the tax is optional. Gemma 3's worst language in my test is Arabic at 1.41x; Hindi and Bengali sit at 1.23x and 1.16x. Same paragraph, same meaning, one-fifth the Bengali markup of Llama 3.1. The difference is vocabulary allocation: Gemma 3 ships a 262k-token vocabulary with real coverage of Indic scripts, while Llama 3.1's 128k vocabulary was built on a corpus where Bengali barely registers. Tokenizer quality is a training-data decision that shows up as a bill.

3. My input-side numbers are smaller than the study's energy numbers — and that gap is the second half of the story. The study measured 7.9x energy for Hindi and 15.3x for Bengali; my token counts give at most 4.7x and 6.3x. The rest comes from the output side: when a model answers in Hindi or Bengali, it generates more tokens to say the same thing, and lower accuracy means longer, more hedged answers. Tokenization is the measurable floor of the penalty, not the ceiling.

What it costs in practice

Token pricing is per-token, so the multiplier is the price multiplier. At any API's per-million-token rate, the same paragraph in Bengali costs 6.3x the English price through a Llama-based endpoint and 1.2x through a Gemma-based one — before the model has generated a single word of reply. The context window shrinks by the same factor: a 128k-token window holds about 128k English tokens' worth of text, but only ~20k tokens' worth of Bengali on Llama 3.1. And the study's other finding compounds it: the languages that cost the most energy also get the least accurate answers. More money, worse output, shorter effective context — the triple penalty is real, and two of its three legs are tokenizer-shaped.

The fix is unglamorous and entirely doable: train tokenizers on balanced multilingual corpora and allocate vocabulary by speaker population, not by training-data convenience. Gemma 3 already did it. The models that didn't are charging Bengali speakers six times more to say the same sentence, and the receipt is printed in the tokenizer file itself.

Limits, stated plainly

My translations are mine, not the Belebele dataset's, and a three-sentence paragraph is one sample, not a corpus — though the single-sentence re-check reproduces the same ordering. I measured token counts, not joules: I have no GPU power metering on this machine, so the energy multipliers (7.9x, 15.3x, up to 180x) are the Michigan team's measured numbers, not mine. And tokenizers are only the input side; the output-side inflation is their finding, confirmed by the arithmetic gap between my numbers and theirs.

Sources: Naihao Deng et al., "The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference," EMNLP 2026 (DOI: 10.18653/2026.emnlp-main.2113); University of Michigan News, "Generative AI uses more energy on non-English text," November 5, 2026; tokenizers: Qwen/Qwen3-8B, google/gemma-3-4b-it and meta-llama/Llama-3.1-8B (Gemma 3 and Llama 3.1 via unsloth mirrors of the official tokenizer files), OpenAI o200k_base via tiktoken. My measurements: November 14, 2026, local run, full token counts in the table above.