We Tested AI Agents With and Without Headron. The Results Reveal a Different Way to Build Reliable AI.
137 of 138 tasks answered correctly. 89% less routing context. 17,397 verification outcomes. Inside Headron's latest benchmarks for AI agents, context engineering, and proof-carrying software. AI development is moving b
137 of 138 tasks answered correctly. 89% less routing context. 17,397 verification outcomes. Inside Headron's latest benchmarks for AI agents, context engineering, and proof-carrying software.
AI development is moving beyond the question of which model is smartest.
A more important question is emerging:
How much better can an existing AI model become when it knows exactly what to inspect, which rules apply, and what evidence is required before declaring its work complete?
That is the question behind Headron, an open-source verification and contextual intelligence system developed by HLIntel LLC.
Headron is not another large language model or coding assistant. It is infrastructure designed to help AI agents navigate software, select relevant context, execute defined checks, and produce verifiable evidence.
Recent benchmarks across multiple models, languages, and repositories are beginning to quantify the impact.
The Numbers: Headron's Recorded Benchmark Results
The latest published results, recorded under Headron's versioned benchmark contracts, include:
| Measurement | Recorded result |
|---|---|
| Claude task benchmark | 137/138 correct with Headron |
| Same benchmark without Headron | 78/138 correct |
| Free-model screening benchmark | 115/138 correct with Headron |
| Same paired screening tasks without Headron | 58/138 correct |
| Context reduction | 89% fewer prompt tokens than whole-tree routing |
| Seeded enforcement tests | 31/31 refused across 20 toolchains |
| Verification outcomes | 17,397 |
| Declared seeded test cases | 863 |
| Graded benchmark answers | 1,937 |
These measurements come from distinct test suites. They are not combined into one overall accuracy score, and they do not represent an independent industry benchmark.
What makes the results interesting is that Headron does not need to modify the underlying model weights to change the quality of the information and constraints supplied to an agent.
1. The Same AI Models, Different Results
One of Headron's most revealing experiments compared three Claude model configurations on 46 tasks each.
The tasks covered diagnosis, repository hygiene, and planning.
Each model was evaluated in two conditions: a blind baseline and a Headron-assisted condition.
| Model | Without Headron | With Headron |
|---|---|---|
| Claude Haiku | 22/46 | 45/46 |
| Claude Sonnet | 23/46 | 46/46 |
| Claude Opus | 33/46 | 46/46 |
| Total | 78/138 | 137/138 |
Across this recorded task set, correct answers increased from approximately 56.5% to 99.3%.
That represents a 42.8-percentage-point improvement.
The underlying principle is important: an AI model's ability to solve a task depends on more than its general intelligence.
It also depends on the quality of its task-specific context.
Consider an agent asked to modify a large codebase. Without structured guidance, it may need to discover the relevant files, infer conventions, identify dependencies, and determine which tests matter.
Headron aims to make those relationships explicit before the agent starts working.
The planning results are especially interesting
Across the three Claude models, the planning subset produced:
- Without Headron: 2/24 correct
- With Headron: 24/24 correct
The hygiene subset improved from 16/30 to 30/30.
These are repository-authored tasks with defined expected answers, rather than measurements of complete autonomous software delivery. Still, they illustrate the value of turning implicit repository knowledge into explicit, machine-readable instructions.
The opportunity is not simply to give AI more context. It is to give AI the right context, at the right moment, with the right constraints.
2. Context Engineering: 89% Fewer Tokens
AI agents often consume substantial context simply figuring out where to begin.
Large repositories contain thousands of files, historical documentation, configuration files, generated artifacts, and dependencies. Much of that information may be irrelevant to the immediate task.
Headron approaches this through selective context routing.
Instead of treating an entire repository as one prompt, the system can identify relevant paths, symbols, contracts, and verification requirements.
The recorded routing experiment reported:
89% fewer prompt tokens compared with whole-tree routing, and 50% fewer than the unassisted comparison.
Headron's documented entry context is approximately 1,644 tokens, with additional information loaded on demand.
This architecture is particularly relevant to AI coding agents operating under context-window limits or usage-based inference pricing.
Routing accuracy across different models
A separate experiment evaluated whether models selected the correct repository command or route.
| Model | Unassisted | Scoped context |
|---|---|---|
| Claude Haiku | 30/76 | 76/76 |
| Claude Sonnet | 30/76 | 76/76 |
| Claude Opus | 31/76 | 76/76 |
| Qwen2.5 Coder 32B | 29/39 | 38/39 |
| Mistral Small 24B | 24/39 | 38/39 |
| Llama 3.3 70B | 23/39 | 38/39 |
| GPT-OSS 120B | 30/39 | 38/39 |
The result is not that every model becomes equally capable.
Rather, models with different architectures and capabilities can benefit from better-defined routing information.
This suggests a broader engineering opportunity: context infrastructure can improve model utilization without requiring the developer to train a new model.
Importantly, the 89% result measures routing context, not every token consumed throughout a complete agent session.
That distinction matters when evaluating actual inference costs.
3. A New Screening Experiment Across Free AI Models
Headron's newer benchmark work also examines whether structured context can improve task performance across accessible AI model providers.
The screening experiment uses 22 tasks and compares answers generated with and without Headron.
Across the 138 paired task instances where both conditions returned answers:
| Condition | Correct | Accuracy |
|---|---|---|
| Without Headron | 58/138 | 42.0% |
| With Headron | 115/138 | 83.3% |
That is a 41.3-percentage-point improvement on the paired sample.
Individual recorded results include:
| Model | Without | With |
|---|---|---|
| Gemini 3.1 Flash Lite | 8/22 | 22/22 |
| Gemini Flash Lite Latest | 9/18 | 17/18 |
| GPT-OSS 120B | 9/13 | 13/13 |
| Gemini Robotics ER 2 Preview | 1/12 | 6/12 |
| Particle 1 | 5/6 | 6/6 |
The benchmark records also distinguish answered tasks from unavailable or incomplete runs. In the screening record, 88 model lanes returned no answer and received no score.
This separation prevents unavailable models or interrupted executions from being mistaken for incorrect answers.
The screening experiment also does not establish lower total token costs. Some Headron-assisted model runs consumed more reported tokens, reflecting the difference between routing efficiency and full-task inference usage.
The key finding is about answer quality on the evaluated tasks.
For developers experimenting with smaller or less expensive models, that is a meaningful direction to investigate.
4. Verification at Scale: 17,397 Recorded Outcomes
Getting an answer is only part of the problem.
An autonomous agent may produce code that appears correct but has not been compiled, tested, checked against policy, or validated against the actual repository state.
Headron addresses this by treating verification as a separate, explicit process.
Its recorded verification outcomes include:
| Outcome | Count |
|---|---|
| PASS | 7,357 |
| REUSED | 1,097 |
| FAIL | 1,278 |
| NOT RUN | 7,665 |
| Total | 17,397 |
The important feature is the distinction between different kinds of evidence.
A passing check is not equivalent to a check that was never executed.
A previously recorded result is not automatically equivalent to fresh verification.
And a successful command does not necessarily prove that every required obligation has been satisfied.
Across these records, 533 verification runs failed and 271 stopped with an unrun check. None was reported clean despite a failed or unrun check.
Verification as a contract
Headron uses a task-to-proof workflow:
Task β Contract β Execution β Proof β Receipt
A task identifies the work.
A contract defines the required checks and constraints.
Execution runs the applicable checks.
Proof records their outcomes.
A receipt connects the evidence to the checked work.
For example:
headron init
headron verify
headron proof scripts/doctor.py --verify
This creates a more explicit boundary between an agent claiming completion and the system establishing what was actually checked.
The longer-term opportunity is proof-carrying software changes: changes that travel with evidence of the obligations they satisfied.
5. Enforcement Across 20 Toolchains
Headron also tests whether declared boundaries are enforced when deliberately challenged.
Its latest published enforcement record includes:
31 out of 31 seeded breaks refused across 20 toolchains.
The repository additionally declares 863 seeded test cases across its broader guard coverage.
These are different measurements. The 31/31 figure describes a specific cross-toolchain enforcement experiment, while the 863 figure describes the larger declared test inventory.
The purpose is to make safety and verification rules executable rather than relying entirely on instructions inside an AI prompt.
That distinction becomes increasingly important as agents gain access to shells, repositories, external services, and automated workflows.
A prompt can describe what an agent should do.
An enforcement mechanism can independently decide whether a requested operation is permitted within its declared boundary.
6. Real Repositories, Multiple Languages, Measurable Verification
Headron's polyglot benchmark expands beyond its own repository.
The test used eight public repositories, including Flask, ripgrep, fzf, Phoenix, Gleam's standard library, and Zod.
Across those repositories, the benchmark examined:
- 2,505 tracked files
- 1,734 routed files
- 119 selected files for verification comparisons
- Multiple language-specific compilers, formatters, and test runners
An important improvement involved respecting each project's actual build configuration.
A Rust source file belonging to a Cargo project should not automatically be judged through an unrelated standalone compilation command.
Similarly, a formatter that a repository never adopted should not be treated as an authoritative project requirement.
Headron's updated routing distinguishes those cases.
In one recorded refinement, FAIL rows decreased from 58 to 23 as checks were aligned with project adoption and execution context. Further changes reduced that count to 17 without losing any of the 88 remaining PASS rows.
These changes are not evidence that the underlying repositories suddenly contained fewer bugs. They demonstrate more accurate classification of what the verification system was entitled to check.
The benchmark also recorded zero tracebacks in its measured comparisons.
This is a practical engineering advantage: verification should reflect the actual project, not assumptions imposed by an external tool.
7. Evidence-Aware Learning Instead of Endless Agent Loops
Another important direction in Headron is the relationship between verification, learning, and reusable computation.
Conventional AI agent workflows often repeat expensive discovery steps.
An agent reads files, identifies dependencies, creates a plan, executes tools, and later may need to rediscover the same relationships.
Headron is developing a more structured alternative built around versioned contracts, bounded tasks, reusable evidence, and deterministic microprograms.
The existing computation registry includes typed operations for finance, reconciliation, and composition.
Its compute.compose operation can connect declared computation nodes, validate dependencies and metadata, and return intermediate results alongside execution order and source references.
That is different from asking a language model to reason through every arithmetic operation or workflow transition.
The architectural direction is to let deterministic programs perform deterministic work, while models handle tasks that genuinely benefit from language understanding and reasoning.
Cross-session learning and validated evidence reuse remain areas of development rather than demonstrated general-purpose autonomous learning capabilities.
But the direction is significant.
An agent should not have to rediscover everything that the software system can already establish, calculate, or verify.
8. The Headron Harness: Infrastructure Around the Model
Headron's broader architecture is designed to operate across existing AI tools rather than requiring a single proprietary model.
The ecosystem includes:
Headron Core: Open-source repository intelligence, contextual routing, verification contracts, and evidence generation.
Headron Harness: A companion runtime and interface for connecting agent workflows to controls and verification.
Mind API and MCP: Integration surfaces intended to expose contextual intelligence and approved capabilities to compatible AI tools.
Microprograms: Typed, bounded operations for calculations, validation, and workflow composition.
The underlying design separates three kinds of work:
- System 1: Fast contextual routing and relevant information selection.
- System 1.5: Deterministic contracts, governance, verification, and evidence.
- System 2: Deeper reasoning when a task requires more extensive analysis.
These are architectural concepts, not claims that Headron has trained its own foundation model.
The aim is to make existing models more useful by improving the infrastructure surrounding their decisions.
9. What These Results Suggest About the Future of AI Agents
AI agents are becoming capable of taking on larger tasks, but larger tasks also introduce more dependencies, more state, and more opportunities for incomplete verification.
Increasing model intelligence alone does not solve every problem involved in operating reliable autonomous systems.
Agents also need to know:
What work is required?
Which information is relevant?
Which actions are authorized?
What checks must run?
What evidence can be reused?
What has actually been proven?
Headron's recorded benchmarks suggest that explicit context routing and task contracts can substantially improve performance on defined workloads.
Its verification results demonstrate a separate capability: preserving the distinction between successful checks, failures, reused evidence, and checks that never ran.
These are complementary improvements.
One helps agents find and use the right information.
The other helps establish whether their work meets the requirements.
Together, they point toward a different way of thinking about AI infrastructure.
Not every improvement needs to come from a larger model, a longer prompt, or another autonomous loop.
Some improvements can come from better software architecture around the intelligence we already have.
What's Next for Headron
Headron's development is moving toward more composable microprograms, evidence-aware computation, adaptive context routing, and reusable task contracts.
The goal is an interoperable infrastructure layer that can support increasingly capable agents while retaining explicit control over what they access, execute, and claim to have completed.
The benchmark results are early, repository-authored evidence rather than a final statement about universal agent performance.
But they establish a useful foundation for further testing.
The future of autonomous software may depend as much on what systems can verify as on what models can generate.
And that is the direction Headron is building toward.
Explore Headron
Open-source repository: https://github.com/HLIntel/Headron
Website: https://headron.tech
Per-model benchmark results: https://github.com/HLIntel/Headron/blob/main/docs/RESULTS.md
Polyglot benchmark: https://github.com/HLIntel/Headron/blob/main/docs/BENCHMARK-POLYGLOT.md
Verification and evidence: https://github.com/HLIntel/Headron/blob/main/docs/CERTIFICATION.md
Headron is developed by HLIntel LLC. Benchmark figures are taken from the project's recorded results under its published versioned contracts.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.