We ran 20 MCP install commands verbatim. Ten started.

A field test by the Anclum team, November 7, 2026.

Every MCP directory has the same table: a name, a one-line description, and an install command. Almost none of them tell you whether that command actually works.

We run one of those directories. So we decided to test ourselves.

We pulled 20 servers from our own catalog, took the exact install command our pipeline extracted from each README, and ran it on a clean Linux host. No tweaking, no "obviously you'd fix the typo first." Just the command, verbatim, on a machine that had Node, Python and a Rust toolchain — Docker and Go were not available.

Ten of the twenty processes came up. Six failed. Four we could not judge.

This is the part where most directories would go quiet. We'd rather show you the numbers, because the failure pattern tells you something useful about the MCP ecosystem right now — and about how much you should trust an install line you found anywhere, including on our site.

How we tested

The rule for "works" was strict:

• Run the command exactly as extracted from the README.
• For npx / uvx / claude ... -- <launcher>: the process must come up and stay alive — stdio servers wait for input, so staying alive is success. We stopped it after a wait and recorded how long it ran.
• For pip: install into a clean virtualenv and confirm the entry point resolves. This is a weaker test — it proves it installs, not that it runs — and we say so in the table.
• For docker / go / cargo: install the toolchain first, then run the documented command.

Two outcomes were recorded separately and not counted as passes or failures: missing host prerequisites (no Docker daemon, no Go toolchain) and runs we could not judge.

The sample was stratified by install method, not chosen by hand: 5 Claude-style, 5 npx, 3 pip, 3 uvx, 2 docker, 2 Go/Cargo.

What happened

So of 20: 10 started, 6 failed, 4 not judged.

The single most important line in that table is the second row. Five of the six failures were not the projects' fault — they were ours. Our text-cleaning step treated _ as Markdown emphasis and deleted it, turning SCAVIO_API_KEY into SCAVIOAPIKEY. The server installed fine and then refused to start, because the variable it was reading didn't exist.

That is a directory bug, not an authoring bug, and we only found it by trying to start the thing.

The failure patterns, concretely

1. Underscores stripped from environment variable names. SCAVIO_API_KEY becomes SCAVIOAPIKEY. The install succeeds; the server exits immediately. A second server failed the same way (VERIFYAX_API_KEY). A third refused to start because three variables it needs — ICTBROADCAST_BASE_URL, ICTBROADCAST_USERNAME, ICTBROADCAST_PASSWORD — appear nowhere in the install line we extracted. We have since changed our cleaner to leave underscores alone — a stray literal underscore on a page is cosmetic, a corrupted variable name is a broken install.

2. Prose leaked into the command. One README's install example was followed by a sentence, and our extractor swallowed it:

claude mcp add tmux -- mcp-tmux defaults to local (project) scope, which is

That's not a shell command. Run it verbatim and you get a syntax error in a fraction of a second. The correct line was simply npx -y mcp-tmux.

3. A trailing period inside the package name. npx umbra-scan. — with the period. npm 404s. Remove one character and the server starts and prints a valid trust score.

4. A "how to try it" example mistaken for an install command. One README shows how to point the MCP inspector at a live API — that's a debugging tip, not installation. Our extractor took it as the install line, so we installed the inspector instead of the server. The process came up and stayed alive, so by our own rule it counted as a pass — a false pass we only caught when reading the command back.

5. A namespace that no longer exists. One README's install path goes through a hosted service whose namespace now returns a 404. This is not fixable from our side and not fixable by the author's README either — the underlying package is gone.

6. Compiling from source is its own category. One Rust server was still compiling when we stopped it at 120 seconds. It may well work on a beefier machine. We recorded it as not judged, because we could not distinguish "slow" from "hung" inside a reasonable window.

What we changed after this test

• The cleaner no longer strips underscores.
• We added a check that rejects install lines containing sentence-like text, and lines where a package name ends in punctuation.
• We mark Docker-requiring servers explicitly, instead of implying you can run them anywhere.

What this means if you're picking an MCP server

Three things worth doing before you spend an evening on a server:

1. Check the variable names. If the README says FOO_API_KEY and the process exits immediately with no message, try the same name without underscores. It's a depressingly common failure.
2. Suspect the install line, not the package. Half our failures installed the package perfectly and then failed on the command around it.
3. Give up at a sensible time. If a from-source install is still compiling after a couple of minutes, that itself is the finding.

Limits of this test

It is 20 servers, on one machine, on one day, with Node, Python and a Rust toolchain available, and Docker and Go unavailable. It is a sample, not a census. A server that failed here might work fine on your machine — especially the two Docker ones and the one we stopped while it was still compiling. We publish the method and the numbers rather than a verdict, because a single "works / doesn't work" badge would be dishonest at this sample size.

What we can say confidently: a README install command is a claim, not a fact, and five of the six failures we saw were introduced by the pipeline that republished the command, not by the project behind it.

We'd rather you knew that before you tried one on our site than after.

Method and per-server results are reproducible from the run log and result file kept with this test. The catalog entry for each server above lists the install command we currently publish; where we found and fixed a bad extraction, the corrected command is what's live now.