DeepSeek V4.1 Flash vs Claude Haiku 5.5: a 17-task benchmark of the two cheapest tiers

Originally published on the QuickSilver Pro blog.

DeepSeek V4.1 Flash and Claude Haiku 5.5 are the two models people reach for when a job runs thousands of times a day: classification, extraction, small code edits, the background calls of an agent. Both are cheap per token. The question is which one is cheaper per finished task, and whether either gets things wrong more often.

So we measured it, on 2026-10-20, through the QuickSilver Pro API.

Update, 2026-11-07: DeepSeek V4.1 Flash's price fell on 2026-11-06, from $0.125 / $0.55 to $0.0558 / $0.2231 per million tokens. Haiku's price did not change. Below, costs are shown at both prices, computed from the same measured tokens; the test itself was not rerun.

How we tested

17 tasks, every one with an answer a script can check:

• 7 Python coding tasks (duration parser, interval merging, LRU cache, strict Roman numerals, an expression evaluator with C-style division, semantic-version ordering, shell-style word splitting). The returned code is run against a fixed set of unit tests; it passes only if every assertion holds.
• 4 extraction tasks (an invoice from an email, attendees from meeting notes, log-line counts, the final state of an order after a chat full of changes). The returned JSON is compared to the expected object field by field.
• 6 reasoning tasks with known numeric or one-word answers (counting arrangements, a pipes-and-tank problem, a weekday two years out, a step-by-step simulation, an elimination puzzle, a digit-sum count).

Each model got the same single user message per task, through the OpenAI-compatible /v1/chat/completions endpoint, non-streaming, max_tokens 8,000, no system prompt and no temperature setting. Each task ran three times, so 51 requests per model. Cost is the billed input and output tokens multiplied by the QuickSilver Pro prices on 2026-10-20. DeepSeek V4.1 Flash thinks before it answers by default, and those reasoning tokens are billed as output, so they are included.

Results

Costs are computed from the measured tokens: the first row at QuickSilver Pro prices on 2026-10-20, the second with DeepSeek V4.1 Flash's price from 2026-11-06.

What stands out:

• Same pass rate, very different bill. Both models passed 50 of 51. Haiku did it for about one seventh of the cost at the prices we tested with, and about a third at Flash's new price. It is cheaper per token on both input and output, and it wrote fewer tokens: Flash averaged 1,730 output tokens per task, 88% of them reasoning the reader never sees.
• Haiku answered faster on every task type. Its median was 2.8 seconds against 9.9 for Flash, and its slowest answers came back much sooner: on the expression evaluator, the shell splitter and the elimination puzzle, Flash usually took 30 to 60 seconds, against 2 to 18 seconds for Haiku.
• The misses were different. Flash gave a wrong number on the elimination puzzle once (the other two runs were right). Haiku's one miss was a response that ended in the middle of the function; the other two runs of that task passed.

Turning off Flash's reasoning

DeepSeek V4.1 Flash can skip the thinking step. On QuickSilver Pro you pass reasoning: {"enabled": false} in the request body; we checked that the response then reports zero reasoning tokens. With reasoning off, Flash became three times cheaper and almost as fast as Haiku, and dropped three tasks: the elimination puzzle twice and the duration parser once. At 2026-10-20 prices it was still about 2.5 times Haiku's cost per task; at the 2026-11-06 price the two cost about the same, $0.13 per 1,000 tasks each.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.quicksilverpro.io/v1",
    api_key="YOUR_QSP_KEY",
)

response = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Extract the invoice number from: ..."}],
    extra_body={"reasoning": {"enabled": False}},
)
print(response.choices[0].message.content)

Which one to use

• Claude Haiku 5.5 for high-volume work with short prompts: extraction, classification, small functions, routing. On this test it matched Flash's accuracy at about a third of the cost of Flash with default settings, and answered faster.
• DeepSeek V4.1 Flash if your prompts are long. Haiku's price rises to $0.125 / $0.625 for any request whose prompt is longer than 200K tokens, while Flash keeps one rate across its 1M-token context. We did not test long prompts, so compare the two prices for your own prompt sizes.
• If you do use Flash for short tasks, try it with reasoning off first. On our tasks it gave up a little accuracy for a much smaller bill and much faster answers. At the 2026-11-06 price it costs about what Haiku does, but Haiku was still faster (2.8 against 3.6 seconds median) and passed two more runs.

Both models are starter models, so the $0.05 free credit a new account gets covers a test like this one: one pass of all 17 tasks on both models cost us about two cents.

Limits of this test

17 tasks and three runs per model is a small sample. One task more or less changes a pass rate by two points, so read 50/51 versus 48/51 as "about the same", not as a ranking. The tasks are short, self-contained and in English; long documents, multi-turn conversations, tool calls and other languages were not tested. Latency was measured from one client on one day and depends on load. Run your own prompts before you switch a production workload: every request appears in your dashboard with its tokens and cost, which makes that comparison easy.

For the full Claude price list, see Claude API pricing in 2026.