How to Cut Claude Opus API Cost With Jev: 4x on Easy Data, 1.8x on Hard

Hightouch published a benchmark this week (581 points on Hacker News) claiming that Jev, a cheap "decision model", gets you Claude Opus accuracy at 4x lower cost if you let it hand the uncertain cases to Opus. On their easy dataset that's true. On their hard dataset, the same setup is only about 1.8x cheaper than Opus alone. If you're trying to reduce Claude Opus API cost, the 1.8x number is the one you should plan around, because your production data is probably not the easy set.

I still think the pattern is worth copying. The headline is the part I'd discount.

One disclaimer up front: I didn't run any of this. Every number below comes from Hightouch's post (Nov 4, 2026), the Hacker News thread about it, and TypeSafe's Jev announcement. It's one company's benchmark on one task with synthetic data.

What was actually tested

The task is identity resolution. You get a profile, which is a bundle of records that supposedly belong to one person, and you decide whether it's actually more than one person (an "over-merge"). Hightouch generated the data with pseudopeople, a University of Washington IHME tool that simulates a fictional US town and spits out census, survey, WIC, Social Security, W-2 and 1040 records with realistic typos and noise.

• Easy set: 1,000 identities, half clean-ish single people, half over-merges.
• Hard set: same town, only the nasty cases. Typos in key fields, twins, Jr./Sr. pairs. The post doesn't give its size.

Jev comes from TypeSafe AI, which calls it a "System One" model. You define the possible outputs up front, and it returns a typed answer plus a calibrated confidence score. TypeSafe lists pricing at $0.042 per million input tokens, with output tokens listed as "FREE (too cheap to meter)". It's in early access. Hightouch never names the maker in the post body (you only find out from the link), so check TypeSafe's page yourself before you build on it.

Jev vs Opus: the numbers per 1,000 identities

Easy set:

Hard set:

$3.86 / $0.88 is about 4.4x. $5.39 / $2.94 is about 1.8x. Same models, same band. Only the data got harder, and the cost advantage fell by more than half.

Why the 4x shrinks to 1.8x

Here's how the hybrid works. Jev scores every identity. Anything with a confidence between 30% and 90% goes to Opus for a second opinion, and everything outside that band keeps Jev's answer.

On the easy set, Hightouch reports 22% of identities landed in that band. That accounts for nearly the whole bill: $0.05 for Jev plus 22% of $3.86 comes to roughly $0.90, close to the reported $0.88.

They don't publish the escalation rate for the hard set, but you can back it out. ($2.94 − $0.06) / $5.39 ≈ 53%. An HN commenter (ezotony) got the same figure. So on hard data, about half of all identities end up paying Opus prices anyway.

The router is working as designed here. When Jev is unsure more often, it escalates more often, and accuracy stays at Opus level (95.3% vs 95.1%) without anyone retuning anything. The catch is that your savings depend entirely on what fraction of your traffic is easy, and a benchmark built half from obvious cases will flatter the ratio.

Jev alone fell from 95.7% to 89.1% on the hard set, so I wouldn't seriously consider running the cheap model unsupervised.

The Luna rows show why calibration matters more than raw accuracy. On the easy set, GPT-6 Luna alone beat Jev alone (97.3% vs 95.7%). But Hightouch found Luna's mistakes came with high confidence, so a band router lets them through. On the hard set, the best Luna + Opus hybrid (10–90% band) reached only 91.6% at $1.15, while Jev + Opus held 95.3%.

The pattern you can copy today

The part I'd keep is confidence-band routing. A cheap model that returns a calibrated probability makes the decision, and only the uncertain middle goes to the expensive model. You don't need Jev specifically. Any classifier that gives you a probability you trust will do.

Here's a sketch. cheap_decide is a placeholder because TypeSafe's announcement doesn't show request/response formats, so wire in whatever your cheap model's SDK returns.

import anthropic

client = anthropic.Anthropic()

LOW, HIGH = 0.30, 0.90  # Hightouch's Jev band; tune on your own labeled data

def cheap_decide(profile: str) -> float:
    """Return P(profile contains more than one person). Plug in Jev, a
    fine-tuned classifier, or any model with calibrated probabilities."""
    raise NotImplementedError

def opus_decide(profile: str) -> bool:
    msg = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=16,
        system="Answer only YES or NO.",
        messages=[{
            "role": "user",
            "content": f"Do these records describe more than one person?\n\n{profile}",
        }],
    )
    return msg.content[0].text.strip().upper().startswith("YES")

def is_over_merge(profile: str) -> tuple[bool, str]:
    p = cheap_decide(profile)
    if p < LOW:
        return False, "cheap"
    if p > HIGH:
        return True, "cheap"
    return opus_decide(profile), "opus"

The band is asymmetric on purpose. In Hightouch's runs, the standalone cutoff they picked for Jev was 60%, not 50%, which tells me its uncertainty doesn't sit neatly around 50%. I'd bet your model's won't either.

Before you ship it:

1. Label 200–500 real examples from your own traffic, not a vendor's synthetic set.
2. Check calibration by bucketing confidence and comparing to actual accuracy. Hightouch found GPT-6 Luna overconfident in the 90–100% bucket, which means a band router would wrongly trust its most dangerous mistakes.
3. Sweep the band and plot cost against accuracy. Hightouch's Jev bands escalated anywhere from 7% (50–80%) to 74% (10–100%) of the easy set.
4. Log the escalation rate in production. It's your real cost multiplier, and it will drift when your inputs change.
5. Price the cheap model per call, then add the escalated share × Opus cost. That's the whole formula. Run it before you believe anyone's "Nx cheaper".

What the HN thread got right

The comments were sharper than the post in a few places:

• Calibration claims were disputed. pcwelder said the claim that Jev's mistakes cluster near 50% "is also incorrect", and bayesnet argued bar charts are the wrong way to show calibration. If calibration is shaky, the band is shaky.
• Cheap models are bad at knowing when they don't know. beefsack said cheap models were "quite bad at knowing even when to call out". The hybrid bets everything on that. Hightouch's version sidesteps it a little by reading a numeric score instead of asking the model whether it needs help. netshade, on the other hand, said Jev "would usually make the right call" when deciding to escalate in their own setup.
• Haiku might be enough. netshade said that once Haiku 5.5 came out, "the cost differences disappeared almost immediately" for their tasks. Hightouch didn't benchmark Haiku or Sonnet, and I wish they had. That's the comparison most Claude users actually care about.
• Do the deterministic work first. jwpapi argued this is "more like compiler work than LLM work". Normalizing names and dates before any model sees them would shrink the uncertain band for free.
• Privacy. vouwfietsman called it "AI-assisted privacy violation, 4 times cheaper". Others pointed out adtech has done identity resolution for years. Fair either way, but if you're merging real people's records, the cost of a false merge goes beyond a wrong row in a dashboard.

My take

The Jev vs Opus result is real, but it's narrower than the title. On hard data you're paying a bit more than half of Opus-only prices for Opus-level accuracy, and that's still a good deal. Just don't put "4x" in your budget proposal. Put your own escalation rate in it.

Are you routing LLM calls by confidence yet? If you are, what share of your traffic actually needs your most expensive model, and did it look anything like 22%?

Sources

• Hightouch, "A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost": https://hightouch.com/blog/llm-judges-identity-resolution
• Hacker News discussion: https://news.ycombinator.com/item?id=50397429
• TypeSafe AI, "Introducing System One Models and Jev": https://typesafe.ai/blog/introducing-system-one-models-and-jev
• pseudopeople: https://www.pseudopeople.org/