Dev.to AI 🤖 Ai 👁 0 📖 2 min read

I Ran 300 Real Jev Calls on a Simulated Robot Fleet. The Bill Was $0.00737.

TypeSafe's Jev came out of stealth this month and every demo I see is the same: browsers, games, toy agents. Jev doesn't chat — it takes structured state and returns typed decisions in 70–500ms, at $0.042 per million inp

TypeSafe's Jev came out of stealth this month and every demo I see is the same: browsers, games, toy agents. Jev doesn't chat — it takes structured state and returns typed decisions in 70–500ms, at $0.042 per million input tokens with output free. Interesting pricing. But nobody was pointing it at anything physical.

So I did. Warehouse robot fleet, incident triage: 41 bilingual incident templates (LiDAR degradation, localization drift, battery faults — the unglamorous stuff), 300 incidents sampled, each API call asking for three simultaneous judgments: escalate to a human (yes/no), owning team (choice), urgency (0–2). All through OpenRouter's decisions endpoint, typesafe/jev-1.13, on September 19.

300 calls, 300 succeeded. p50 0.53s, p95 0.81s. Mean input 585 tokens. Cost per decision: $0.0000246. The entire run cost $0.00737 — less than a cent. I keep staring at that number.

Then the fleet math, because that's the part that actually matters. 10,000 robots × 48 decisions/day × 30 days = 14.4 million decisions a month. Jev: $354/month. GPT-4o-mini on public pricing math: ~$1,814/month. 5.1x cheaper on cost. I did not measure GPT-4o-mini's triage quality — this is a cost ratio, not a quality claim, and I'm saying it up front because the fake-demo discourse around Jev is already exhausting.

I also ran the obvious counter-experiment: self-hosted ModernBERT-base on a 2-core CPU. p50 169ms — about 3x faster than Jev's 527ms. But one label per call instead of three judgments, and you need exemplars, tuning, and a VM you babysit. Break-even lands around 977K decisions/month (roughly 678 robots at 48/day). Below that, the API is cheaper and you skip the ops entirely. Jev's edge was never raw speed. It's zero training, zero labeling, zero infrastructure.

What this actually shows: sub-second, three-judgments-per-call triage at ~$25 per million decisions, with no training data. What it doesn't show: production accuracy. My incidents are simulated, so 91.3% "agreement" is against template labels, not real failures. No real robot hardware, no long soak test, no calibration study on the confidence scores. Those are next.

Everything is in the repo: runnable scripts, per-call data, charts, and a 60-second video of the cost ticker. I also maintain a curated list of Jev resources — evals, real demos, and a papers section that currently says "none" because TypeSafe hasn't published one and I'm not padding it.

Code and data: https://github.com/robokrunch/jev-physical-ai
Curated Jev resources: https://github.com/robokrunch/awesome-jev

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.