On-device AI agents got real this month. Where a small local model beats a big remote one.
Three releases from October point the same way: Liquid AI LFM2.5-24B-A2B: an open mixture-of-experts model built for on-device agents. 24B total, 2B active per token. Liquid measured 184 tokens/s on an Apple M5 Max.
Three releases from October point the same way:
- Liquid AI LFM2.5-24B-A2B: an open mixture-of-experts model built for on-device agents. 24B total, 2B active per token. Liquid measured 184 tokens/s on an Apple M5 Max.
- NVIDIA Nemotron 3.5 Bolt E4B: a compact model for local assistants, about 4B effective parameters.
- Liquid AI LFM3: a family of small text, vision, audio and decision models, each picked by what it does and where it runs.
The "2B active" part is the trick. The model knows a lot (24B parameters), but each token only uses a small part of it, so it runs at a speed a laptop or a good phone can handle.
Where a local model wins
Not on raw quality. A frontier model in a data centre is still better at hard reasoning. A local model wins on everything around the answer:
1. Privacy by construction. The text never leaves the device. For a parking app, think of a driver's location history, or an owner's photos of their home driveway. "We don't send it" is a stronger promise than "we send it, but securely".
2. It works with no signal. I wrote about a trip where the signal dropped halfway through. Basement car parks are exactly where you lose network. A feature that needs the cloud fails at the moment it's needed.
3. No per-call cost. Once it's on the device, a thousand calls cost the same as one. That changes which features are worth building: things like checking every input or classifying every photo.
4. Latency without a network round trip. For small, frequent decisions this adds up quickly.
Where it doesn't
- App size and memory. Even small models are hundreds of MB to a few GB. That's a big download for a parking app.
- Battery and heat. Fine for occasional calls, harder for continuous ones.
- Hard reasoning. Planning a multi-step task or writing long text is still better done remotely.
- Updates. A model shipped inside an app updates when the app does.
The pattern I'd use: local first, remote when it matters
Use the small local model for the many small decisions, and the big remote model for the few hard ones:
| Task | Where |
|---|---|
| "Is this photo a parking spot?" | Local |
| "Does this message mention a payment problem?" | Local |
| Voice to text for search | Local |
| Explain a disputed charge | Remote |
| Plan a multi-stop trip | Remote |
The local model can also decide when to call the remote one, which is a nice use for the small decision models in LFM3.
What I'd tell you before you start
On-device AI isn't a smaller version of cloud AI. It's good at different things: private, offline, free per call, and instant. Pick the features where those matter, and keep the hard thinking in the cloud.
Sources: Liquid AI blog (Oct 13 and Oct 27, 2026) and NVIDIA model card (Oct 23, 2026), via "AI Model Releases: October 2026 Tracker", Digital Applied. Speed figures are the vendors' own.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.