Cheapest LLM for product-review sentiment: Mistral Small at $0.000019 a success

On 300 Amazon product reviews, Mistral Small (2603) is the cheapest of nine models that clears an 88% pass bar and can be shown to match the baseline, GPT-6.1 Sol. It passes 97.7% of the tasks against 98.7% for Sol, at $0.000019 per successful task against $0.000319: 94.0% less.

This is an easy task, and the table shows it: all nine models clear the bar, so the question is only which one costs least. Two near neighbours deserve a straight reading. Claude Haiku 5.5 costs $0.000023 per successful task and answers faster (p95 336 ms against 912 ms for Mistral Small). GPT-6 Luna costs $0.000024 and passes 98.0%. If latency or the last fraction of a point matters more than a few thousandths of a cent, those are the picks.

The run. 300 reviews drawn with a fixed seed from the Amazon Polarity test split (Apache-2.0). One task: reply with positive or negative. Temperature 0, up to 2 retries when the reply is not a valid label, every attempt's cost counted, prices read from OpenRouter on 2026-11-05. Total spend for all nine models: $0.22.

Pass bar. 88% correct. A model "clears" only if the lower end of its 95% Wilson interval is at or above the bar. "Matches" means an exact one-sided paired test of "no more than 5 points worse than the baseline" on the tasks both models answered.

Where the retries went. qwen3.8-flash needed 5 retries and Haiku 2, so their prices carry a small retry share (1.5% and 1.4% of what each spent). Mistral Small needed none. On an easy task the retry tax is small; on the 77-label banking-intent run, deepseek-v4.1-flash spent 31.3% of its money on retries.

What a miss looks like. Mistral Small got 7 of 300 wrong, and 3 of those 7 the baseline also missed. One is "I love mandatory fees. The regulatory cost recovery fee rocks!", a sarcastic review the data labels positive and Mistral Small called negative. Labels like that put a ceiling under every model, which is why a pass bar below 100% is not a lowered standard.

Check it or re-run it. The 300 rows, the prompt, the prices and the re-run script are on the verdict page, so you can run the same comparison with your own OpenRouter key (the full nine-model run cost $0.22) and see where your numbers land. Prices move monthly; the page notes the day it was last re-checked: https://clearbar.grain64.com/verdicts/amazon-sentiment/?utm_source=devto&utm_campaign=devto-sentiment

Want the same verdict on your own data? Import 10-100 of your input/expected pairs, add your current model, and the workspace names the cheapest model that clears your bar, with cost per successful task, retries counted, and the failures it will have. Free during early access; you pay OpenRouter for the model calls with your own key.

Which task should the next verdict cover: extraction, summarisation, or classification of your own labels? Say so in the comments.

From Clearbar, by Grain64.