Replaying 147 Claude Code & Codex Sessions to Tune autoCompactWindow (and Catching a 3x Token Overcount)

My first script for checking local Claude Code token usage summed raw assistant lines in ~/.claude/projects/**/*.jsonl and overcounted session 4baee81b by 3.0x (33 JSONL usage rows for 11 actual API turns). Whenever a single turn emits text plus multiple tool_use blocks, Claude Code writes separate streaming records sharing the same message.id, and every record repeats the cumulative prompt token count. Codex CLI has a similar quirk where event_msg (token_count) rows repeat identical info.total_token_usage.total_tokens snapshots.

After deduplicating by message.id across 147 local Claude Code and Codex sessions on my machine (3,197 raw records collapsing to 2,284 real API turns, 255.79M input tokens, 43.9% cache hit rate), two things jumped out:

1. 13.4% of turns drove 47.8% of weighted token cost. Specifically, the 307 turns that ran above 200K context accounted for nearly half of all cache-weighted input token-equivalents (where cache_read_input_tokens is weighted at 0.1x and uncached input at 1.0x), peaking at 467,835 tokens in session 78fd79c6.
2. Early tool outputs carry massive Context Rent. A 12K-token file read or git diff output on turn 3 of a 45-turn session sits in the prompt window for 42 subsequent turns (12,000 * 42 = 504,000 token-turns of context rent). Worse, when compaction finally triggers, the agent often immediately re-reads the exact same files on the next 3 turns because the summary dropped active file paths.

Replaying local transcripts with compactsim

I built compactsim as a zero-dependency Node.js CLI to audit ~/.claude/projects and ~/.codex/sessions offline and run counterfactual compaction simulations against your actual turn history.

Run it directly without installing anything:

npx github:princejain756/compactsim

Here is the real output on my 147-session corpus:

CompactSim v0.1.0 - Context Compaction & Token Waste Simulator
Corpus: 147 sessions (123 Claude Code, 24 Codex) | 2,284 API turns (3,197 raw JSONL usage records, 1.40x raw overcount)

Baseline Token Profile:
- Total input tokens: 255.79M (143.49M uncached @ 1.0x, 112.30M cache-read @ 0.1x, 43.9% cache hit rate)
- Weighted input token-equivalents: 160.79M tok-eq
- Peak session context: 467,835 tokens (session 78fd79c6)
- Turns > 200K context: 307 turns (13.4% of turns driving 47.8% of weighted token cost)
- Turns > 400K context: 59 turns (2.6% of turns driving 15.7% of weighted token cost)
- Mid-session cache-bust spikes (>25K uncached jump): 760 spikes across 147 sessions

Counterfactual Policy Replay:
1. Baseline (recorded transcripts):      160.79M tok-eq |   0.0% saved |  9 compactions |  8.7m wait
2. autoCompactWindow = 400,000 tokens:   138.92M tok-eq | -13.6% saved | 10 compactions |  9.8m wait
3. autoCompactWindow = 250,000 tokens:   114.93M tok-eq | -28.5% saved | 11 compactions | 11.9m wait
4. autoCompactWindow = 150,000 tokens:    89.49M tok-eq | -44.3% saved | 17 compactions | 18.2m wait
5. Sliding tool-output eviction (15t):   136.67M tok-eq | -15.0% saved |  9 compactions |  8.7m wait

How the counterfactual simulator works

When you lower autoCompactWindow (for example from the default window down to 250,000), you do not get free savings:

• The turn immediately following compaction pays a 1.0x uncached re-prefill cost on the compacted summary plus system prompt (~22K tokens).
• If the compaction summary drops active working context, the agent re-reads files on the next 1 to 3 turns.

compactsim models both the uncached compaction re-prefill penalty and the empirical post-compaction file re-read tax observed in your transcripts. On my corpus, autoCompactWindow = 250,000 hits the sweet spot: it cuts weighted token usage by 28.5% (160.79M down to 114.93M token-equivalents) while adding only 2 extra compactions across 147 sessions.

Code and zero-dep CLI are MIT-licensed at https://github.com/princejain756/compactsim.