Your model doesn't need to pass the bar exam. It needs to parse a log file.

Intro

Every few weeks, another frontier model claims a new benchmark, better reasoning, longer context, higher scores on tests built to measure how humans think. None of that is irrelevant, but it's solving a problem a huge share of enterprise workloads don't actually have. Parsing a structured log line, validating a field against a schema, classifying a support ticket into one of six categories - none of that was ever going to need a model that can debate philosophy or pass the bar exam. It needs a model that's fast, cheap, and right, every time, on a narrow task. That's a different design goal than the one the frontier race is optimizing for, and it points toward small, specialized, locally hosted models instead of the next big release. Below are four places where the gap between "benchmark-optimal" and "production-optimal" actually shows up. (Illustrative composites, not case studies from a specific client.)

Latency budgets don't care how smart the model is

Picture a fraud-detection pipeline calling a hosted frontier model on every transaction. It works fine in testing. Under peak load, p95 latency starts spiking, not because the model reasons badly, but because every call is a network round trip through someone else's queue, competing with every other tenant's traffic on that provider that day. A distilled model of a fraction of the size, running on the same box as the service that calls it, doesn't have a network hop to blame. Latency stays predictable in the low tens of milliseconds because there's no shared queue to wait behind.

Your data boundary is only as strong as your last API call

A team building on patient or financial records sends that data to a third-party endpoint for every inference call. It works, until a compliance review asks a simple question: where does this data physically go, who retains it, and for how long. If the answer is "a provider's infrastructure, under their retention policy," that's a boundary the team doesn't fully control and can't fully audit. A small model running inside their own perimeter turns that into a non-question. There's no external boundary to explain, because the data never left.

A pricing or deprecation decision made by someone else is not a risk you control

A workflow built on top of a hosted model API works well for a year, until the provider raises prices, changes rate limits, or deprecates the exact model version the workflow was tuned against. None of that is a bug in the team's code. It's a business decision made by a company they don't work for, and it forces an unplanned re-engineering effort on someone else's timeline. A locally hosted, version-pinned model doesn't have a roadmap owned by another company deciding when it stops being supported.

A model fine-tuned on your schema beats a generalist prompted around it

Ask a general-purpose frontier model to validate rows against a company's actual database schema, and it will usually get it right, and occasionally hallucinate a plausible-sounding field name that doesn't exist, because it's reasoning from general knowledge about how schemas tend to look, not from the ground truth of this one. A small model fine-tuned on the company's real schema isn't guessing at a pattern, it's seen the exact structure it's being asked to validate against.

Every one of these is really the same story with a different failure mode

A task with a narrow, well-defined shape got handed to a general-purpose tool built to be good at everything, at the cost of being cheap, fast, and predictable at any one specific thing.

Build vs. integrate: when a small local model actually wins

If the task is narrow and repetitive with a stable shape, log parsing, schema validation, ticket classification, a small specialized or fine-tuned model tends to win on latency, cost, and data boundaries. If the task involves genuine ambiguity, novel reasoning, or synthesizing loosely related domains on the fly, a frontier model is still the better tool. A useful test in between: could a domain expert write down the rules for what "correct" looks like on this task? If yes, that's a strong signal a small local model can be trained or fine-tuned to do it more cheaply and predictably than a general model can be prompted to.

Where's the line for you? At what point does reaching for the biggest available model stop being ambition and start being the wrong tool for the job?