Three things that break when you point Claude Code at your own model

Claude Code can talk to any endpoint that speaks the Anthropic Messages API. That sentence makes it sound like a one-line change. In practice, the first time you point it at a model you serve yourself, it fails in a few predictable ways, and each one looks like a broken model until you know where to look.

We ran a full session this way: Claude Code driving Qwen3 Coder Next, an 80B mixture-of-experts coding model, served by vLLM on a single rented A100 80 GB. The complete walkthrough of the 80B coding model on one A100 has every command, screenshot and number. This post is the short list of what broke, why, and what fixed it.

1. Tool calls come back as a 400

Claude Code is an agent. Almost every request it sends carries tool definitions: read a file, run a command, edit a component. A plain vllm serve does not accept those, and vLLM answers requests that carry tools with an HTTP 400. From the Claude Code side it looks like the model is broken.

The fix is two flags on the server, the same pair the model card recommends:

--enable-auto-tool-choice --tool-call-parser qwen3_coder

The parser has to match the model family. With them on, the session went through plan mode, wrote files, installed packages, ran npm run build and kept editing, tool calls included.

2. The model name does not match

This is the one that bites on the very first prompt. Claude Code sends Claude model names by default, and it quietly uses a second, smaller model for background work. An endpoint that serves one open model has no idea what claude-something is.

On our endpoint the manager checks the requested model and accepts only the workload slug, its model name, default or no model at all. Anything else gets a 400 with "Model '…' is not available on this endpoint". So every model role Claude Code knows about has to point at the same name: the main model, the small background one, and the sonnet and opus aliases too, because the /model picker and opusplan still send claude-* names otherwise.

That is four environment variables plus the base URL and the token. The exact block is in the step-by-step post with the Claude Code environment variables, ready to copy.

3. The base URL and the auth header

Two smaller traps sit next to the model name.

The base URL stops at the workload. Claude Code appends /v1/messages itself, so if you paste the full route, /v1/messages ends up in the path twice.

For the key, ANTHROPIC_AUTH_TOKEN goes out as Authorization: Bearer. ANTHROPIC_API_KEY goes out as x-api-key and also works, but it overrides a subscription login, and interactive Claude Code asks you once to approve the custom key. If you set both with different values, our manager rejects the request with a 401. Pick one.

Why the model fits on one card at all

Two separate ideas, and it helps to keep them apart. The weights are a 4-bit AWQ quantization that cyankiwi published on Hugging Face, which brings 80B parameters down to roughly 40 GB and leaves room on the 80 GB card for the KV cache. The mixture-of-experts design is what makes it quick: all 80B weights stay in memory, but each token only computes through about 3B of them.

The server was configured for the model's full native context, 262,144 tokens, with the KV cache stored in FP8 so that context fits next to the weights, and at most two sequences in flight. That last number is honest: this is a one-developer setup.

What it does not show

The run proves the plumbing, not the quality. A landing page is not a large legacy refactor, the model is a 4-bit quant whose quality loss we did not measure, and we did not compare it against hosted Claude on the same task. The 256K context was configured, not exercised: partway through the session it had used 31.7K tokens. On the A100 we only have a smoke test, so there is no throughput figure for it in the write-up. And this is Claude Code driving Qwen, not Claude, so agent behavior depends on how well the model follows Claude Code's tool protocol.

Cost has the same shape of caveat. Paying per GPU hour ($1.79/h for the A100 that day) only beats paying per token when several developers or agents keep the card busy.

Credits

The model is Qwen3 Coder Next from the Qwen team, the 4-bit version is cyankiwi's, vLLM did the serving, and Anthropic made Claude Code work with compatible endpoints. We wired them together.

The full command with what every flag does, the env block, the GPU telemetry from the session and the list of limits are in the InferenceKey write-up of the Claude Code on one A100 experiment. If you would rather follow a guide than read a story, there is also Deploy your own code assistant in the docs.