Dev.to AI 🤖 Ai 👁 0 📖 7 min read

I Trimmed My Agent's Tool Results to Five Fields. It Solved More, Not Fewer.

A reader named izgorodin left a comment on my piece about the 200-tool menu, and it changed the question I thought I was answering. He had run the numbers himself: forty turns of a triage helper whose only tool returns f

I Trimmed My Agent's Tool Results to Five Fields. It Solved More, Not Fewer.

A reader named izgorodin left a comment on my piece about the 200-tool menu, and it changed the question I thought I was answering. He had run the numbers himself: forty turns of a triage helper whose only tool returns five GitHub issues, and by turn twenty the tool results were 99.1 percent of everything sent, about 326,000 tokens, thirteen times the cost of the menu itself. Keeping only the last six turns flattened the call at around 99,000. Trimming each result to the five fields the helper actually reads kept the whole history and ended turn forty at 36,934. Then he wrote the sentence that matters: his replay called no model and used a fixed answer, so it measured how big the window gets, not whether the helper still finds what it needs.

That is the question I spent this week answering. Same twelve triage jobs, run twice: once with the full issue text coming back, once with each result cut to the five fields (the number, the title, the labels, the comment count, and the first 300 characters of the body). A real model did the work, a small one I run locally, and every job was scored on whether it found the right issue and the right label. I expected the trim to cost something. I was wrong about the direction.

My claim before the numbers: trimming the results would save about half the tokens per turn and cost a few solved jobs, because sooner or later the helper would need something the trim cut. The first half was right. The second half was backwards.

Problem 1: the trim did not cost a single solved job

Across both runs, the full results solved 9 of 24 jobs. The trimmed results solved 17 of 24. That is 37.5 percent against 70.8, and the gap is not a rounding error in one direction: the trim won 9 to 4 in the first run and 8 to 5 in the second. Look at the twelve jobs one by one and the trim never loses a job the full text wins. The two arms agree on seven jobs (both solve four, both fail three), and on the other five the trim wins or ties every time.

Three jobs beat the model in both arms, and they are worth naming because they set the limit. On the Linux dependency guide, the Windows 11 question, and the misspelled error dialog, the model answered the crash-on-launch issue every single time, in both arms, in both runs. Those are small-model limits, not trim damage. The trim cannot be blamed for what the full text also fails, and the fact that the failures are identical in both arms is the cleanest evidence in the whole run that the trim is not hiding anything load-bearing.

Problem 2: the full text has a favourite issue, and it is always the crash

The full arm's failures have a shape, and once you see it you cannot unsee it. On five of its failed jobs, the model answered issue 412, the crash on launch, with the label "bug" — whatever the job actually asked. The job about garbled dates in a CSV export? 412, bug. The job about the install guide missing a Linux step? 412, bug. The job about a typo in an error dialog? 412, bug. It even went back and read 412 a second time before answering it, as if checking its work.

The trimmed arm does the same second look just as often, about once per job in both arms, but it answers the right issue afterwards. The difference is not that the trim hides the crash report. It is that five short results sit side by side and all five stay visible, while five long ones let the most dramatic report swallow the rest. A small model reading a long list does not weigh five issues. It remembers the loudest one. Cutting the noise did not just save tokens. It changed what the model could pay attention to, and attention was the scarce thing, not information.

Problem 3: the full text can eat the whole window just by looping

The ugliest full-arm result is not a wrong answer. It is no answer at all. On two jobs the model listed the issues, listed them again, and listed them again, six turns of the same call, and burned 10,063 tokens on one job before the turn cap stopped it. The first time I ran this, that loop pushed the prompt past the model's whole window and the run died outright. I added a rule that a repeated call comes back as a one-line note instead of the full dump again, the same rule in both arms, and the run survived, but the loop still cost the job.

The trimmed arm never needed a fourth turn on any job. Every job ended in three: list, one second look, answer. The full arm averaged close to four turns and needed six on its worst jobs, and each of those extra turns carried another thousand-plus tokens of history with it. The token bill per turn tells the same story from the other side. Turn one is identical in both arms, about 179 tokens. Turn two is 1,749 with full results and 759 trimmed. By the third turn the full arm is sending about 1,900 tokens a turn and the trimmed arm about 1,080. The average across the run is 1,421 tokens per turn against 673, and 5,475 per job against 2,020. The trim cuts the bill by more than half per turn and almost two thirds per job, and the jobs it saves are the ones where the full text was already losing.

Problem 4: the second look is not the price of the trim

izgorodin asked for a miss signal alongside the solved rate, and he was right to: a trim that quietly sends the helper back for the full text every time has not saved anything, it has just moved the cost. So I counted every time the helper asked for one issue in full. Full results: 0.88 second looks per job. Trimmed results: 0.92. That is the same number, inside the wobble between runs, and it is the number that surprised me most. The trim did not drive the helper back for more. The helper goes back about once per job in both arms, usually to look once more at the issue it is about to answer, and the full text costs the same second look for the same reason.

That kills the natural objection in both directions. The trim is not free because the helper never needs the cut material: it goes back just as often. And it is not expensive because of the second looks, because the second looks cost the same either way. The saving is in the list itself, paid once per turn, and the second look is a fixed cost of doing the job carefully, not a tax on trimming.

Problem 5: the honest limit is the size of the model

All of this is one small model on twelve jobs. A bigger model would solve more jobs in both arms (the three jobs that beat this one in both arms would probably fall), and the gap between the arms would likely shrink. The absolute rates, 38 percent and 71 percent, belong to this model and these twelve jobs, and anyone quoting them as general truths about agents is over-reading a small run. Twelve jobs is also small enough that the run-to-run wobble is a whole job per arm. I ran it twice for that reason, and the direction held in both runs, but the direction is the finding, not the percentages.

The other limit is the jobs themselves. My issue bodies run a hundred-odd words, shorter than real reports. Real bodies are longer, messier, and more repetitive, which would widen the token gap, not narrow it. But it would also give the trim more to cut, and I have not measured that. The five fields are also izgorodin's five, chosen for a triage prompt that cites numbers and suggests one existing label. A different prompt reads different fields, and its trim would cut somewhere else.

So the rule I am keeping is narrower than the one I went in with. Trim the result to the fields the prompt actually reads, and measure the solved rate before you argue about it, because the cost you expect (the helper going back for what you cut) may not exist, and the benefit you did not expect (the helper deciding better with less noise) may be the whole story. The menu was never the bill. The results were. And the results, it turns out, were also the distraction.

What do you trim from your tool results, and did you check whether the solved rate moved when you did? I would like to hear the number, either way, and what the helper went back for when it went back.

P.S. Everything I write stays free — that part doesn't change. But if this one was worth something to you, you can buy me a coffee ☕ (one-time or monthly, both work). It covers the API credits and hosting behind these experiments — and honestly, it's also the nudge that gets the next one started on the evenings I'd otherwise skip it. — Salam

Buy me a coffee

AheadKit on Gumroad — Beginner-Friendly Books & Packs

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.