Reading an LLM price card without fooling yourself

Every few weeks someone posts a table of "cheapest LLM APIs" and it gets a thousand upvotes. Two weeks later half the rows are wrong, and the half that are still right were never comparable in the first place.

The problem isn't that people lie about prices. It's that a price card is not a single number, and a comparison table forces it to be one.

Here is the checklist I run before I let a published price into a spreadsheet.

1. Anchor the number to a URL and a date

A price with no source is a rumour, and a price with a source but no date is a rumour with a footnote. Vendors change price cards quietly — no changelog, no announcement, just a different number on the same page.

So the minimum unit of record is three fields, not one:

model            | price | source_url                  | observed_on
gpt-class-mini   | ...   | https://vendor/pricing      | 2026-09-23

If you can't fill source_url with the vendor's own pricing page — not a blog post about the pricing page — you don't have a price. You have a claim.

2. Input, output, and cached input are three different prices

Output tokens almost always cost more than input tokens, often several times more. Cached input, where offered, costs less than fresh input. A table with one column per model is silently picking one of these three and hoping your workload matches.

It usually doesn't. Two realistic shapes:

• Summarisation / extraction: long input, short output. Input price dominates. A model with cheap input and expensive output can be the cheapest thing you run.
• Generation / agents: short prompt, long completion, many turns. Output price dominates, and the "cheap" model from the first case becomes the expensive one.

Same two models, opposite verdicts, and nothing about the price card changed.

3. Do the arithmetic against your own traffic shape

You don't need a benchmark for this, you need your own logs. Pull a day of real requests, get the median input and output token counts, and compute:

def cost_per_call(p_in, p_out, tok_in, tok_out, cached_ratio=0.0, p_cached=None):
    """Prices in $ per 1M tokens. Returns $ per call."""
    p_cached = p_in if p_cached is None else p_cached
    billed_in = tok_in * ((1 - cached_ratio) * p_in + cached_ratio * p_cached)
    return (billed_in + tok_out * p_out) / 1_000_000

# one day of traffic, not one prompt
def monthly(calls_per_day, **kw):
    return cost_per_call(**kw) * calls_per_day * 30

Ten lines, and it settles arguments that comparison tables cannot. The ranking it produces is yours; the ranking in the table is the author's workload wearing your name.

4. Watch the unit, not just the number

"Per million tokens" is the convention, but tokens are not a universal unit — each vendor's tokenizer splits your text differently. For English prose the spread is small enough to ignore. For code, JSON, non-Latin scripts, or anything with heavy punctuation, it isn't. If your payloads are structured, tokenize a real sample with each vendor's own tokenizer before you compare per-token prices.

5. Multimodal is where the comparison quietly breaks

Images and audio get converted to tokens at a vendor-specific rate that depends on resolution, tiling, and sometimes on a "detail" flag you pass in the request. Two vendors can publish the same per-million-token price and bill you very differently for the same photo.

For anything multimodal, the only honest comparison is an empirical one: send the same ten real inputs to each candidate and read the usage field in the response. That's a fifteen-minute experiment that beats any table, including this article's.

6. Treat "under one dollar" claims as a starting point, not a finding

Headline claims about million-token context or sub-dollar pricing are usually true under stated conditions — a specific tier, a batch endpoint, a cached-input assumption, a promotional window. The conditions are the interesting part, and they're the part that gets dropped in the retelling.

I keep a running note of which published price cards I've actually opened and checked against the vendor's own page, and what the fine print turned out to say: what the published price cards actually prove. The pattern that shows up most often isn't a false number — it's a true number attached to conditions almost nobody meets.

For multimodal specifically, where the token conversion rates do most of the damage, I keep a separate shortlist of what survives an actual usage-field check: cheap multimodal API picks.

The short version

Record the URL and the date. Split input from output. Run your own traffic through ten lines of arithmetic. Tokenize your real payloads. For multimodal, measure instead of reading.

None of this is hard. It's just not what a ranked table gives you, and a ranked table is what everyone shares.