I built a research agent that tries not to sound more confident than it should

Most research agents I’ve encountered share a rather annoying tendency: they hallucinate a citation (or exaggerate the credibility of a weak source) and still present the answer with complete confidence. Consequently, I often find myself spending a significant portion of my time “using” these agents simply to manually verify whether the sources genuinely support the claims they made.

I got tired of that, so I built ResearchAgent.

It’s a focused open-source agent designed for technical and architectural research inquiries. You provide it with a specific task or question to address.

“Compare the top 3 open-source vector databases for a startup with 10k QPS and tight cost constraints”

…and it returns a structured answer with:

• A clear recommendation
• Only real sources it actually fetched
• A calibrated confidence score
• Explicit limitations ranked by severity

Design choices that mattered

Instead of one big reasoning blob, it runs through explicit stages:

1. Planner – breaks the question into concrete steps
2. Executor – runs search, page fetching, optional code execution, MCP tools
3. Verifier – applies domain reputation heuristics and checks whether the page actually supports the claim
4. Synthesizer – produces the final answer, confidence score, and limitations.

The confidence score is not merely a subjective assessment. The synthesizer meticulously evaluates a concise checklist that encompasses the number of sources, the consistency between them, their recency, and whether the original constraints were effectively met. Consequently, sources of low authority or insufficient quality are filtered out early, minimizing the likelihood of them influencing the final score.

It also supports full local execution (DuckDuckGo + Ollama) if you don’t want to use paid APIs, and has optional human approval gates before planning and synthesis.

Why the stages help

At first the extra structure feels like friction. But the first time something goes wrong, being able to look at the plan, the raw evidence, and the verification scores separately is much better than staring at a wall of chain-of-thought and guessing where it failed.

Try it

Repo is here:

https://github.com/spidervirus/ResearchAgent

It includes progressive examples (from a plain LLM call all the way to the full agent) and a short tutorial if you want to see how each piece works.

I’m still improving the confidence calibration and want to move toward claim-level confidence later. Feedback from people who actually use research agents is very welcome.