LLM Cost Tracking: Per Request, Feature and User (2026)

I am the founder of PromptUnit. This post first appeared on the PromptUnit blog, where it is kept up to date.

LLM cost tracking means recording what every model call costs, together with the feature, user, model and environment that caused it, so you can see where AI spend comes from and act on it. Without it, cost work is guesswork.

Most teams start with a monthly invoice from OpenAI or Anthropic and no visibility beneath that number. The invoice tells you what you spent. It does not tell you which feature drove the spike last Tuesday, which model is responsible for 70% of your bill, or whether your costs grow in step with your users or faster.

This guide covers what to track, a cost function you can copy, how to get cost per user, the tools that can do it for you, and what to do when costs move. Provider prices, spend limits and tool features were checked on October 30, 2026.

The Core Metrics to Track

Cost per feature

The most actionable dimension for engineering teams. When you can see that Feature A costs $0.008 per call and Feature B costs $0.47 per call, you know where to focus.

This requires tagging every LLM call with the feature that generated it, in your logging code or with a request header if you use a proxy. LLM cost attribution by feature shows what OpenAI and Anthropic can and cannot group by, and how to tag requests.

Cost per user (or tenant)

For SaaS products, per-user LLM cost is the base of your unit economics. If your average customer generates $12/month in LLM costs but pays you $29/month, your AI margin is thin. If one customer generates $200/month in costs, the monthly invoice will not tell you who it is. The per-user section below has the queries.

User-level cost tracking also supports fair-use policies, per-plan rate limits and pricing decisions.

Model distribution

What share of your calls goes to each model? If 90% of calls go to your most expensive model, check whether every one of them needs it. If you recently moved some traffic to a cheaper model, model distribution tells you whether the change is working as intended.

Tracking model distribution over time also surfaces drift: as new features ship, they may default to an expensive model without going through a cost review.

Task type breakdown

Which task types are most common in your traffic? What is the average cost per task type? Tasks that are both high-volume and simple are the first place to test a cheaper model.

Cost per environment

Separate tracking for production, staging and development is basic hygiene. A cost spike in development (a developer testing with a long context window, a test suite making more calls than expected) should not stay invisible until the invoice arrives.

Cache hit rate

Cached input is billed at a fraction of the normal input price, so the share of input tokens read from the cache moves your bill. OpenAI's prompt caching guide suggests dividing total cached tokens by total input tokens and grouping by user, workspace or day. A drop often means something near the start of the prompt began changing between calls. Prompt caching on OpenAI and Anthropic covers how each provider caches.

Latency by model

Cost and latency are related. Smaller models are often faster. Tracking latency per model per task type shows user-facing performance next to cost when you decide which model a feature should use.

Setting Up Cost Monitoring

Option 1: Log every call with one cost function

The basic approach: every LLM call is logged with its cost and metadata when it is made. The cost has to come from the usage object the provider returns, priced per model, and it has to handle cached tokens.

OpenAI and Anthropic report cached tokens differently. In OpenAI's Responses API, input_tokens includes the tokens read from the cache and the tokens written to it. In Anthropic's Messages API, input_tokens counts only the tokens after the last cache breakpoint, and cache reads and writes come in separate fields. A function that ignores this overstates the cost of cached calls on OpenAI and understates it on Anthropic.

from datetime import datetime, timezone

# USD per 1M tokens, standard tier, from each provider's pricing page on October 16, 2026.
# OpenAI rows are for prompts up to 272K input tokens.
# Claude Haiku 5.5 row is for prompts up to 100,000 tokens.
# Claude cache_write is the 5-minute cache; 1-hour cache writes cost more.
PRICES = {
    "gpt-6.1-sol":       {"input": 2.00, "cached": 0.10, "cache_write": 2.50,  "output": 10.00},
    "gpt-6-sol":         {"input": 2.00, "cached": 0.20, "cache_write": 2.50,  "output": 10.00},
    "gpt-6-luna":        {"input": 0.10, "cached": 0.01, "cache_write": 0.125, "output": 0.50},
    "claude-sonnet-5-5": {"input": 2.00, "cached": 0.10, "cache_write": 2.50,  "output": 10.00},
    "claude-haiku-5-5":  {"input": 0.10, "cached": 0.01, "cache_write": 0.125, "output": 0.50},
}


def cost_usd(provider: str, model: str, usage) -> float:
    p = PRICES[model]  # KeyError on purpose: an unknown model must not be logged at $0
    if provider == "openai":
        # Responses API: input_tokens includes cached and cache-write tokens.
        # output_tokens includes reasoning tokens, which are billed as output.
        details = usage.input_tokens_details
        cached = details.cached_tokens or 0
        written = getattr(details, "cache_write_tokens", 0) or 0
        uncached = usage.input_tokens - cached - written
    elif provider == "anthropic":
        # Messages API: input_tokens counts only tokens after the last cache breakpoint.
        cached = usage.cache_read_input_tokens or 0
        written = usage.cache_creation_input_tokens or 0
        uncached = usage.input_tokens
    else:
        raise ValueError(f"no usage mapping for {provider}")
    return (
        uncached * p["input"]
        + cached * p["cached"]
        + written * p["cache_write"]
        + usage.output_tokens * p["output"]
    ) / 1_000_000


def log_call(provider: str, model: str, response, feature: str, user_id: str) -> None:
    log_event({
        "ts": datetime.now(timezone.utc).isoformat(),
        "provider": provider,
        "model": model,
        "feature": feature,
        "user_id": user_id,
        "usage": response.usage.model_dump(),  # raw counts, for later analysis
        "cost_usd": cost_usd(provider, model, response.usage),
    })

The prices come from OpenAI's pricing page and Anthropic's pricing page. Our price pages for GPT-6.1 Sol, GPT-6 Sol, GPT-6 Luna, Claude Sonnet 5.5 and Claude Haiku 5.5 show today's numbers.

An example of why the cache fields matter: a Claude Sonnet 5.5 call with 10,000 input tokens read from the cache, 2,000 uncached input tokens and 500 output tokens costs (2,000 x $2.00 + 10,000 x $0.10 + 500 x $10.00) / 1,000,000 = $0.0100. The same call with no cache would cost (12,000 x $2.00 + 500 x $10.00) / 1,000,000 = $0.0290.

Keep the price table in one place and update it when prices change. Long prompts are priced higher on some models (over 272K input tokens on OpenAI's newer models, over 100,000 tokens on Claude Haiku 5.5), so add those tiers if your prompts get that long.

This approach works, but every call site must pass the feature and the user. It is easy to miss one.

Option 2: A wrapper that every call goes through

A better approach is to make the wrapper the only way your app calls a model, so no call is made without attribution:

class TrackedClient:
    """The only path to the model, so no call goes unlogged."""

    def __init__(self, client, feature: str):
        self.client = client
        self.feature = feature

    def respond(self, user_id: str, model: str, **kwargs):
        response = self.client.responses.create(model=model, **kwargs)
        log_call("openai", model, response, self.feature, user_id)
        return response


summaries = TrackedClient(openai_client, feature="ticket-summary")

Option 3: A proxy or gateway

A proxy or gateway sits in the request path and records every call, so there is no logging code at each call site. You change the API base URL and send the feature or user as request headers. Helicone's AI Gateway, the LiteLLM proxy and PromptUnit work this way; the tools table below shows what each one records. The trade-off is one more service between your app and the provider.

How to Track LLM Cost per User

Provider usage reports are built around your account structure, not the users of your app. Anthropic's usage report, for example, groups by API key, workspace, model and a few service settings, not by the user a request was made for. Per-user cost has to come from your own logs or from a tool that reads a user id from each request.

1. Pass a user id with every call. Use your internal user id (a UUID from your database), not an email address. In your own logs it is a column, as in the log_call function above. If you use a gateway, send it the way the gateway expects: Helicone reads the Helicone-User-Id header, LiteLLM reads the x-litellm-customer-id header or the request's user field, and Langfuse takes a userId on traces. One user action in an agent can make several model calls, so log the same id on all of them. The multi-agent cost spiral explains why those calls add up.

2. Group by user and day. With calls stored in a table such as llm_calls (ts, user_id, feature, model, cost_usd), this Postgres query gives daily cost per user for the last 30 days:

SELECT user_id,
       date_trunc('day', ts) AS day,
       count(*)              AS calls,
       round(sum(cost_usd)::numeric, 4) AS spend_usd
FROM llm_calls
WHERE ts >= now() - interval '30 days'
GROUP BY user_id, day
ORDER BY day DESC, spend_usd DESC;

3. Compare with what the user pays. Join to your users and plans to see AI cost as a share of each paying user's price:

SELECT c.user_id,
       u.plan,
       p.monthly_price_usd,
       round(sum(c.cost_usd)::numeric, 2) AS ai_cost_30d,
       round((100 * sum(c.cost_usd) / p.monthly_price_usd)::numeric, 1) AS pct_of_price
FROM llm_calls c
JOIN users u ON u.id = c.user_id
JOIN plans p ON p.name = u.plan
WHERE c.ts >= now() - interval '30 days'
  AND p.monthly_price_usd > 0
GROUP BY c.user_id, u.plan, p.monthly_price_usd
ORDER BY pct_of_price DESC
LIMIT 20;

A user on a $29 plan with $12 of AI cost in 30 days spends 41% of their price on AI alone (12 / 29). Free users have no price to divide by, so list them separately, sorted by cost. Pick a threshold that fits your margins and review everyone above it. AI cost per user covers how to set that threshold from your revenue per user.

Best LLM Cost Tracking Tools in 2026

Each option below reports LLM cost. They differ in how they get the data, how finely they can split it, and whether they can stop spending or only report it. Every cell comes from the tool's own documentation, checked October 16, 2026. PromptUnit is our product, and its row lists only what this post documents.

Sources: Langfuse cost tracking, users, tags and alerts; Helicone header directory, custom properties, rate limits and alerts; LiteLLM customers, tag budgets, budgets and spend tracking; OpenAI spend limits, Anthropic workspaces, Gemini API billing.

How to choose: if you need to stop one user's or one feature's spending, Helicone's cost-based rate limits and LiteLLM's budgets do it in the request path. Langfuse alerts you but leaves the call alone. The provider consoles cap a whole project or workspace. For a closer look at two of these, see PromptUnit vs Helicone and PromptUnit vs Langfuse.

Alerting Setup

Cost tracking is only useful if anomalies surface in time to act. The providers cover the first alert type below: OpenAI sends spend alerts at amounts you set on its Limits page, and Anthropic can alert when a workspace's spending reaches thresholds you set. The other two need your own logs or a tool from the table above.

Absolute cost threshold

Alert when spend in the current billing period exceeds a fixed dollar amount. Set it at 80% of your expected monthly budget.

Alert: Monthly LLM spend has exceeded $4,000 (80% of $5,000 budget)

Percentage increase vs rolling average

Alert when daily or hourly cost is well above the rolling 7-day or 30-day average. This catches spikes that do not hit absolute thresholds but show that something changed.

Alert: LLM spend in the last 1 hour is 3.2x the 7-day hourly average

This type of alert is most effective for catching unexpected traffic increases, runaway loops, or new features calling expensive models at high volume.

Per-feature cost anomaly

Alert when a specific feature's cost per call rises sharply. A 5x increase in cost per call for a feature usually points to a prompt change, a context window change, or a model change that was not cost-reviewed.

Alert: Feature "document_analysis" average cost per call increased 4.7x
(from $0.023 to $0.108) in the last 24 hours

Cost Anomaly Detection

Common causes of unexpected cost spikes:

New feature launched without a cost estimate. A feature that generates long outputs, uses a long context window, or chains several model calls can cost many times more per call than a typical feature. If cost estimation is not part of the feature review, the first signal is the invoice.

Prompt change increased output length. A prompt that used to produce 200-token responses now produces 800-token responses after an edit. Output tokens cost more than input tokens on every major provider: five times on OpenAI's GPT-6 models (for prompts up to 272K tokens) and six times on GPT-5.6 Terra and GPT-5.4, five times on Anthropic's current Claude models, and from five times on Gemini 3.8 Flash to about eight times on Gemini 3.5 Flash-Lite. OpenAI bills reasoning tokens as output, and Google's output price includes thinking tokens. So output length has an outsized effect on cost.

Context window growth over time. For multi-turn conversation features, context grows with each turn. A conversation that started at 500 tokens per call may be at 8,000 tokens after 20 turns. If your application does not trim or summarize conversation history, per-call cost keeps growing over a session.

Model change without a cost review. Moving a feature from GPT-6 Luna to GPT-6.1 Sol, or from Claude Haiku 5.5 to Claude Sonnet 5.5, multiplies the per-token price by 20 on both input and output ($0.10 to $2.00 per million input tokens, $0.50 to $10.00 per million output tokens). If cost monitoring does not flag the change, it stays invisible until the invoice.

Runaway agent loop. Agent workflows that call tools and retry can make far more model calls than planned. A bug that makes an agent loop, or a task harder than the retry budget assumed, can multiply the call count.

For a way to turn calls and tokens into a monthly bill, see OpenAI API Cost Too High? Six Fixes, Ordered by Effort. For what defaulting to one expensive model costs beyond the invoice, see The Hidden Cost of Defaulting to Your Flagship Model.

Budget Enforcement

Tracking tells you what happened. Enforcement stops it from happening again.

Provider spend limits

All three large providers let you cap monthly spend, but none of them does it per API key:

• OpenAI: a monthly spend limit for the whole organization or for one project. With "Enforce a hard limit" on, requests over the limit get a 429 error with the code organization_spend_limit_exceeded or project_spend_limit_exceeded. Enforcement is not instant, so recorded spend can slightly exceed the limit. (OpenAI spend limits)
• Anthropic: a monthly spend limit per workspace, set in the Claude Console under Settings > Workspaces. It can be lower than the organization's limit but not higher, and the Default Workspace cannot have its own limit. (Anthropic workspaces)
• Google: a monthly spend cap per project, set on the Spend page in AI Studio. Google marks it as experimental and says spend can run past the cap for about 10 minutes while billing data catches up. Billing accounts on paid tiers also have a monthly cap ($250 on Tier 1, for example); when it is reached, all linked projects pause until the 1st of the next month. (Gemini API billing)

The risk: one cap covers everything in the project or workspace, so a single runaway feature can block every other feature that shares it. Giving each critical feature its own project or workspace limits the damage.

Per-feature limits

Enforce limits at the feature level in your application or gateway. A feature expected to make 10,000 calls a day at $0.05 per call should stop accepting new requests after 12,000 calls. LiteLLM's tag budgets and Helicone's per-property rate limits do this in a gateway.

Per-user spending limits

For multi-tenant or consumer-facing products, enforce a per-user LLM spend limit. A single power user who generates 100x the average LLM traffic is an economics problem. Helicone can limit spend per user with Helicone-RateLimit-Policy using u=cents and s=user, and LiteLLM can give each customer a max_budget. Or check the per-user total from your own logs before each call.

Pre-call cost estimation

Before making a model call, estimate the expected cost from the input token count and the model. If the estimate exceeds a threshold, reject the request, use a cheaper model, or ask for explicit confirmation.

From Tracking to Cost Management

LLM cost management is the loop that turns tracking data into lower or more predictable bills. It has four steps:

1. Track. Log every call with its cost, feature, user and model, using the cost function above or a tool from the table.
2. Alert. Set a provider spend alert at 80% of the monthly budget, and a spike alert on your own data for cost per hour and cost per call by feature.
3. Cap. Put hard limits at the provider for each project or workspace, and per-user or per-feature budgets in your code or gateway.
4. Review. Once a week, sort features and users by spend and look at the top three of each. For each one, decide whether the prompt can be shorter, whether caching would help, whether a cheaper model holds up on that task (test it on your own traffic first), or whether the plan price needs to change.

LLM Observability: The Metrics That Actually Drive Decisions covers the other metrics worth watching next to cost.

Where PromptUnit Fits

PromptUnit checks each request, and when a cheaper model is just as good for that request, it uses that model; otherwise the request goes to the model you asked for. Setup is two settings: point your app's AI API base URL at it and add your PromptUnit key. Your own provider key stays. For cost tracking, the dashboard shows the cost of every request and breaks cost down per model, and per feature when you tag requests with the optional x-promptunit-feature header (letters, digits, underscore, hyphen and dot, up to 64 characters). It also groups requests into the steps of your app or agent and shows what each step costs.

PromptUnit has no per-user view, so keep per-user cost in your own logs with the queries above. In Privacy mode only what billing needs is stored (the models, tokens, cost and response time), so feature names and steps are not kept.

See Also

• OpenAI API Cost Too High? Six Fixes, Ordered by Effort
• The Hidden Cost of Defaulting to Your Flagship Model
• LLM Cost Attribution by Feature
• AI Cost Per User
• PromptUnit vs Helicone
• PromptUnit vs Langfuse

Try It Free

See where your AI budget goes. PromptUnit's dashboard shows every request with the model that answered and what it cost. It is free until it has saved you money, then it takes 20% of what it saves.

Start free.

FAQ

What is LLM cost tracking?

Recording what each model call costs, together with the feature, user, model and environment behind it. The provider invoice gives you one total. Tracking per request shows which feature or user drives the spend and whether cost grows faster than usage.

What is the difference between LLM cost tracking and LLM cost management?

Tracking tells you where the money went. Cost management is what you do with that data: alerts before the budget runs out, hard caps at the provider or in a gateway, and a regular review of the top features and users by spend.

How do I track LLM cost per user?

Send your internal user id with every call, compute each call's cost from the usage object the provider returns, and store both. Then group by user and day in SQL and compare each user's 30-day cost with the price of their plan. Gateways such as Helicone and LiteLLM can group by a user id header for you.

Can I set a hard spending limit on OpenAI, Anthropic or Google?

Yes, but not per API key. OpenAI sets monthly hard limits per organization or per project, Anthropic sets monthly spend limits per workspace, and Google lets you set a monthly spend cap per project in AI Studio, a feature it marks as experimental. Checked October 16, 2026.

How much more do output tokens cost than input tokens?

For prompts up to 272K tokens, five times on OpenAI's GPT-6 models and six times on GPT-5.6 Terra and GPT-5.4. Five times on Anthropic's current Claude models. On Google it runs from five times (Gemini 3.8 Flash) to about eight times (Gemini 3.5 Flash-Lite). Checked November 5, 2026.