Dev.to AI 🤖 Ai 👁 0 📖 7 min read

How many tools should an MCP server have? I measured 5 to 100 in Claude Code, Codex and Gemini

Most advice about MCP server design says keep the tool count small. I read 91 sources: 38 say design fewer tools, 20 give a number (mostly 10–20), and only 10 measured tool count against accuracy at all. Then I measured

How many tools should an MCP server have? I measured 5 to 100 in Claude Code, Codex and Gemini

Most advice about MCP server design says keep the tool count small. I read 91 sources: 38 say design fewer tools, 20 give a number (mostly 10–20), and only 10 measured tool count against accuracy at all. Then I measured it: 1,932 runs, three clients, 5 to 100 tools. On today's top models accuracy didn't move, not with 100 tools, not with look-alikes, not even with vague names. What moves is cost, and how much depends on how your client loads tools, not on the model. Since 2026 all three clients hide most of your tool list from the model by default, which makes tool names matter in a new way.

MCP server best practices nobody measured

Before measuring anything, I read 91 sources on MCP server design, every one of them in full: 21 official docs, 28 vendor blogs, 20 independent blogs, 2 forum threads and 20 papers. 38 of them say to curate fewer tools, 22 say to keep the visible set small through search or deferred loading, and 8 recommend code mode, where one or two tools run code. None say to expose everything.

20 sources give a number: 5–8 (Workato), 5–15 (Phil Schmid), under 10 per namespace and under 20 (OpenAI), 10–20 (Gemini, Speakeasy), 30–50 (Claude docs), "40 is the knee" (Archestra). Only 3 of those 20 numbers come from the source's own measurement. Two of the most repeated claims don't hold up. "Models break at 30 tools" traces back to the RAG-MCP paper, which counted MCP servers in the prompt, not tools. "Cursor won't take more than 40 tools" stopped being true when the cap was removed in January 2026.

Of the 91 sources, 46 are opinion and 8 cite someone else. 37 measured something, but only 10 measured tool count against accuracy: 7 papers, GitHub's Copilot (40 → 13 tools for +2–5 points), LangChain and one informal test on a 1.7B model. The rest measured tokens or compared MCP with a CLI, which isn't a tool-count measurement. Even on grouping the advice splits: 15 sources recommend workflow tools, 15 grouped tools, and 3 one tool per operation, and those 3 argue from permission prompts, not accuracy.

How I measured it

I built a fake server with 100 realistic file and document tools, with honest names and schemas. Some of them are deliberately confusable, like pdf_compress, pdf_optimize and image_compress. It also has a grouped variant, with 7 tools covering 50 operations, and a vague-names variant. None of the tools touches a file: the server's call log is the judge.

The agents got 18 tasks with known answers, 10 of them ambiguous, at N = 5, 10 … 50, 75 and 100 tools, two times each. The clients were Claude Code (Opus 5.5) with tool search on and off, Codex CLI (GPT-6.1-sol) and Antigravity (Gemini 3.1 Pro), each on its own subscription. All measurements are on GitHub: the report, results.csv and the raw log of every run.

How many MCP tools is too many?

Every run with separate tools was fully correct, from 5 to 100 tools, in all three clients and in both Claude Code modes. That includes every ambiguous task and the 50 extra look-alike tools at 75–100. So on Opus 5.5, GPT-6.1-sol and Gemini 3.1 Pro (October 2026), "fewer tools = more accurate" didn't show up in this range, which matches the survey's caveat that degradation depends heavily on the model.

Claude Code's tool search never hid the right tool. Every run made exactly one ToolSearch call, as its first action, and selected tools by exact name. The model sees all tool names up front and picks between look-alikes from the names alone, before it reads any schema.

What MCP tools really cost in Claude Code, Codex and Gemini

Input tokens, averaged over 36 runs per cell (18 tasks × 2). Every one of these runs was fully correct, so the table shows cost only.

Client How tools reach the model First call: 5 / 50 / 100 tools Per run: 5 / 50 / 100 tools One more tool (first call / per run)
Claude Code (Opus 5.5), tool search on (default) Names only; schemas loaded on demand with ToolSearch 3.9k / 4.6k / 5.4k 13.8k / 17.4k / 19.9k ~15 / 79 tokens
Claude Code (Opus 5.5), tool search off Every full schema in the prompt 4.1k / 11.7k / 20.2k 9.3k / 27.9k / 48.6k 169 / 429 tokens
Codex CLI (GPT-6.1-sol) Code mode: schemas never in the prompt; the model searches them with code 13.2k flat 45.3k / 61.4k / 78.2k 0 / 408 tokens
Antigravity (Gemini 3.1 Pro) Names only; schema file opened on demand 3.8k / 4.1k / 4.3k 15.7k / 20.5k / 18.6k ~5 tokens / flat

Tool search is the switch that matters in Claude Code. It's on by default: only tool names go into the prompt, and the model loads the schemas it needs with one ToolSearch call. That extra call makes it dearer than loading everything below about 19 tools, and cheaper above: 59% less input per run at 100 tools. It's off behind a proxy or on Azure Foundry, with older models, with ENABLE_TOOL_SEARCH=false, or with --tools "". Fewer tokens didn't mean a cheaper run, though: at list price a run with search on cost $0.047 against $0.038 with it off at 100 tools, because cached input is cheap and the search adds a call. What search saves is context, not money.

The trap I fell into: my isolation flag switched off Claude Code's tool search, and I first reported the "search off" numbers as normal. A single flag changed the first-call cost by 4x.

Grouping MCP tools: cheaper in one client, worse in another

Grouping MCP tools: accuracy and input tokens per run, 50 separate tools vs 7 grouped tools, in four client setups

With every schema loaded (Claude Code, search off), 7 grouped tools cost what 14 separate ones do, at 100% accuracy. With tool search on, Claude Code's default, grouping barely matters: 16.1k input tokens per run against 17.4k for 50 separate tools. Grouping pays off only where every schema is loaded upfront.

Antigravity is the one place in the whole experiment where accuracy dropped, to 89%. Gemini skipped reading the schema of the multi-operation tool, guessed operation names such as delete, help and fake_op, and ran a wrong one 3 times.

Do names decide? Vague names under deferred loading

For this variant I kept the same 100 tools and the same descriptions, and gave the tools careless names: optimize_file, optimize_document, process_image, combine_files, handle_pages. Accuracy held: 276 of 276 runs were correct, and no wrong tool was executed. The descriptions decided the answer.

But wherever the model sees names first, the names decided the path to that answer. In Claude Code with search on, honest names let the model load exactly the schema it needed in 69 of 72 runs; with vague names, in 7 of 72. It fell back to keyword searches, the schemas loaded per run went from 1.6 to 7.6 at 100 tools (23 in one run), and input grew by 21%. Gemini opened the wrong tool's schema first in 34 of 72 runs, and in 10 runs it probed wrong tools with empty arguments. My server rejected those empty calls; on a real server, a tool with no required arguments would have run. Gemini's input grew by 38% at 100 tools.

Where every description is already in the prompt, vague names changed nothing: Claude Code with search off didn't notice them. Good descriptions rescue bad names only where the model can see them. Codex searches the definitions with code, so names barely mattered there either.

The names that misled most often: "3 JPGs into one PDF" led to combine_files, which actually merges PDFs; "shrink a PNG" led to optimize_image; and pptx and docx to PDF got mixed up between convert_file and convert_document.

MCP tool naming: dots, prefixes and the 64-character budget

Names that every client can call use letters, digits, _ and -, up to 64 characters. The Claude API rejects dots, and some clients rename dots on their side. Those 64 characters are also shared: Claude Code calls your tool mcp__<server>__<tool>, so a long server name eats into the budget. One more surprise from the runs: Gemini's API rejected tool schemas with integer enums in one eager-loading run.

So what should you do?

  • Don't cut tools for accuracy's sake on frontier models; cut or defer them for cost and context, and check which of your users' clients loads them eagerly.
  • Name tools by action and object, not by generic verbs: names are now the first thing a model reads, and vague ones cost extra reads and risk a wrong call.
  • Write descriptions a model can tell apart: that's what kept accuracy at 100% with look-alikes and vague names.
  • Don't group blindly: test in the clients your users run.
  • Measure your own server. This worklore story walks through the same benchmark step by step, and your agent can run it against your server.

Limitations

Fake tools and one domain (files). Three frontier models only: smaller and older models weren't tested, and that's where the measured sources in the survey report drops. Direct tasks with no long reasoning alongside, so context crowding wasn't measured. Vague names were tested with good descriptions only, not with both vague. Token counts include each client's overhead. Some sizes weren't run in every lane, and the Codex vague-names lane has 60 of 72 runs because of a usage limit. And it's one point in time: October 2026.

Did you choose your tool count and names by measurement or by advice? What did you measure?

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.