Per-Token Price Is Only Half the Story: Modeling Real LLM API Costs for a Marketplace

Every LLM pricing table answers the same question: what does a million tokens cost? It's a useful number, but it's the wrong unit for a marketplace. A marketplace doesn't buy tokens. It runs listings, matches, moderation passes and fraud checks, and each of those calls hides costs that never show up in the headline rate.

This post treats LLM pricing the way you'd treat any other infrastructure cost: build a model of the workload, find the multipliers, then optimize the parts that actually move the bill. All figures come from the October 2026 pricing comparison published by marketplace development agency Journeyhorizon, which you can read in full in their LLM API pricing guide for marketplaces.

Step 1: Load the price sheet, but read it correctly

Here's the October 2026 snapshot (standard short-context prices per 1M tokens, input / output):

Two things jump out when you look at this like an engineer rather than a shopper:

1. Output is the expensive direction. Across providers, output tokens cost four to six times more than input tokens. Your max_tokens setting is a budget control, not just a UX choice.
2. Tier spread dwarfs provider spread. Budget and flagship tiers differ by 20x to 100x. Comparing Anthropic vs OpenAI vs Google at the same tier barely matters next to choosing which tier handles each task.

Also note the price sheet has a shelf life. Gemini 4 Argon is scheduled to go from $2/$10 to $4/$20 on December 1, 2026, and Gemini 3.8 Flash from $0.75/$3.75 to $1.50/$7.50 on January 1, 2027. Treat prices as config, not constants.

Step 2: Model the workload, not the request

Take the worked example from the source: a skilled-trades service marketplace.

requests_per_day      = 500
llm_calls_per_request = 3   # intake summary, provider matching, fraud screening
input_tokens/request  = 6,000
output_tokens/request = 1,500

daily:   3,000,000 input  +   750,000 output
monthly: 90,000,000 input + 22,500,000 output

Plug that into the price sheet and you get roughly:

Run the premium numbers by hand and you'll notice that output is more than half of the $405 bill, even though there are four times fewer output tokens than input tokens. That's the 5x output multiplier at work.

Now scale it. At 10,000 requests a day the same workload is about $400/month on a budget model versus about $8,100 on Claude Sonnet 5.5. At 50,000 requests a day it's roughly $2,000 versus $40,500. The gap that looked like a rounding error at launch becomes a line item your CFO asks about.

Step 3: Add the multipliers the price sheet doesn't show

This is where most estimates go wrong. Here's a checklist of cost drivers to add to your model:

• [ ] Retries and timeouts. Failed calls still burn tokens. If 3% of calls retry, your bill is at least 3% higher than modeled.
• [ ] Reasoning tokens. "Thinking" tokens are billed at the output rate but aren't in the visible response. A 300-token answer can bill several thousand tokens.
• [ ] Long-context surcharges. Gemini 3.1 Pro rises to $4/$18 above 200K tokens; Claude Haiku 5.5 rises fivefold to $0.50/$2.50 above 100K. One oversized context window can quietly reprice a whole feature.
• [ ] Regional and premium modes. US-only inference is billed at 1.1x; Opus 5.5 fast mode at 2x.
• [ ] Rate limits and fallbacks. You'll need backup models and orchestration logic. That infrastructure isn't in any API price.
• [ ] Traffic shape. Marketplaces aren't chatbots. Flash sales, seasonal peaks and viral listings multiply calls overnight.

A simple way to encode this:

def monthly_llm_cost(base_cost, retry_rate=0.03, reasoning_factor=1.0,
                     region_multiplier=1.0, peak_factor=1.0):
    return base_cost * (1 + retry_rate) * reasoning_factor \
                     * region_multiplier * peak_factor

It's crude, but it forces you to put a number on each assumption instead of pretending they're zero.

Step 4: Pull the levers that actually move the bill

Prompt caching

Anthropic bills cached reads at 10% of the normal input rate. In the source's example, caching a 1,500-token system prompt on Claude Sonnet 5.5 drops the input bill from about $180 to about $52 a month. Two caveats: cache writes cost 1.25x the input rate, and the default cache lives for five minutes. Caching pays off for hot, repeated prefixes, not for one-off prompts.

Batch APIs

OpenAI, Anthropic and Google all offer a 50% discount for queued, non-real-time work, with results within 24 hours. Recommendation refreshes, analytics summaries and non-urgent moderation belong here.

Model cascading

Route easy work to a budget model (Claude Haiku 5.5, gpt-6-luna, Gemini Flash-Lite) and escalate only hard cases. Sending 70% of traffic to budget and 30% to premium takes the $405 example down to about $135 a month, roughly a two-thirds cut.

def route(task):
    if task.kind in {"summarize_intake", "classify_listing"}:
        return "budget"
    if task.confidence < 0.7 or task.kind == "fraud_review":
        return "premium"
    return "budget"

Prompt diet

Trimming redundant instructions and parameterizing content can shrink prompts by 30–50%. But because of the output multiplier, capping response length is usually the single biggest win.

Step 5: Decide with total cost of ownership

Before you commit to a provider, run through four questions:

1. What does a wrong answer cost? A bad moderation call damages trust; a bad match hurts satisfaction. That sets your minimum tier.
2. What's your real volume? Prototype, measure a week of real traffic, then extrapolate. Guesses are always low.
3. What's the switching cost? A cheap provider with an unstable roadmap can cost more if you have to migrate in a hurry.
4. What's the full TCO? Infrastructure, fallback logic, error handling and the engineering hours spent optimizing all count.

There's no universally cheapest provider. Budget tiers like Claude Haiku 5.5 and gpt-6-luna win on raw rates, and open-weight models such as DeepSeek can be cheaper still, but the right answer depends on your workload. Test two or three before deciding.

Takeaway

The per-token price is an input to your cost model, not the model itself. Capture volume, call count, output ratio, retries, reasoning tokens and traffic spikes, then attack the bill with routing, caching, batching and output caps. Do it before launch, because architecture decided on day one decides whether AI features scale profitably.

If you want a second pair of eyes on your marketplace's AI spend, Journeyhorizon's AI cost optimization service is built for exactly this kind of audit.