Dev.to AI 🤖 Ai 👁 0 📖 4 min read

I summed every span in my agent trace. The token count came out exactly double.

I built a small incident triage agent to work out where its tokens went. It answers on-call alerts in three steps with two tools, search_logs and fetch_runbook. The token counts in this test are synthetic. I estimated t

I built a small incident triage agent to work out where its tokens went. It answers on-call alerts in three steps with two tools, search_logs and fetch_runbook.

The token counts in this test are synthetic. I estimated them from request size so they grow the way real usage grows. That was enough to show two mistakes that are easy to make with real agents.

The first one showed up when I added up the tokens for a single run. I got 101,464. The right number was 50,732.

The provider bill cannot tell you which step cost money

A usage page splits spend by model and by day. My agent spends tokens per step, per tool call and per run.

When the spend goes up, I want to know which step pushed it. Was it the log search, a retry, or the long system prompt? A daily total cannot answer that.

So I record usage on every LLM call as a span attribute, with the OpenTelemetry GenAI names:

  • gen_ai.usage.input_tokens for the prompt side of one call
  • gen_ai.usage.output_tokens for the reply

The GenAI conventions are still in Development status. Before you build a dashboard on these names, read one real span from your own instrumentation and check what it actually writes.

My trace counted the same tokens twice

My run is an agent span. Under it sit three step spans, and under each step sit the LLM call spans.

The usage lives on the LLM spans. I had also set the run total on the agent span, the way the GenAI conventions let an agent span carry usage next to its children.

When I summed every span that had usage, I got 101,464 tokens. The LLM spans alone added up to 50,732. The agent span repeated its children, so my naive sum was exactly 2x.

The fix is a leaf rule. Count a span only if it carries usage and no span below it does. Then group those leaf spans by the step, the tool call and the run above them.

There is an open issue in the OpenTelemetry GenAI conventions repo about how to aggregate token usage. Until it settles, I enforce the leaf rule myself.

Most of the tokens were input the loop sent again

Once I had per-call numbers, the growth was plain to see. The first call in the run used 1,078 tokens. The call in step 3 used 16,610. No call returned more than 61 output tokens.

Three things go out again on every call:

  • The system prompt and tool schemas: they never change between calls.
  • Tool output: in step 1, search_logs returned 400 lines of payment gateway timeouts. That text joined the message history and stayed there for every later call.
  • Retries: in step 2 the model returned truncated JSON for the fetch_runbook arguments, so the loop resent the whole context with a short correction.

A 2026 study of agent token costs (arXiv 2604.22750) found the same shape at a much larger scale. Across eight frontier models on SWE-bench Verified, agentic coding tasks used about 1,000x more tokens than code reasoning and code chat. Input drove the cost, and runs of the same task varied by up to 30x.

One step took 77.9% of the run cost

The second mistake is reading spend by day. So I priced every leaf call with a dated price table and ranked the steps by their share of the run cost.

Step 2 of the first run took 77.9%. Its first call was the first to carry the full log dump, and its retry sent that context a second time.

The retry itself cost less than I expected: 9.2% of what both test runs cost together. Almost all of its input hit the prompt cache, 16,384 of 16,486 tokens. A model with no cached rate would bill that same retry in full.

So the first thing I trim now is tool output. One search_logs call pushed 400 lines into context, and nothing in my loop ever took them out.

I check the budget inside the loop

A monthly bill tells you about a bad run weeks later. I wanted the agent to stop itself the same day.

So I check a per-run token budget after every LLM response, before the next tool call. In my test the budget was 40,000 tokens. The first run crossed it at step 3. The second run finished at 34,200 and passed.

This check has a limit. A check after the response cannot stop the call that crosses the line, and my first run ended 10,732 tokens over budget. If you need a hard cap, check before the call: reserve the next call's input plus its maximum output against what is left.

Behind the in-loop check sits a threshold alert on daily tokens per model. It catches slow drift that no single run crosses.

What I would tell myself before tracking agent tokens

  • Sum leaf spans only: an agent span that repeats its children doubles your count.
  • Check the attribute names: the GenAI conventions are still in Development, so read a real span first.
  • Rank steps before days: in my demo, the top step named the tool output to trim.
  • Count retries as cost: each one resends the whole context.
  • Put the budget in the loop: and check before the call if you need a hard cap.

I wrote the full version, with the rollup script, the dated price table and where the budget check sits in the loop, in my token usage tracking write-up.

Have you found a step in your agent that cost more than the rest of the run combined? For me it was the step right after a log search, which carried 400 lines of logs and paid for them again on every call after it.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.