Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 2 min read

On-device AI agents got real this month. Where a small local model beats a big remote one.

Three releases from October point the same way: Liquid AI LFM2.5-24B-A2B: an open mixture-of-experts model built for on-device agents. 24B total, 2B active per token. Liquid measured 184 tokens/s on an Apple M5 Max.

Three releases from October point the same way:

  • Liquid AI LFM2.5-24B-A2B: an open mixture-of-experts model built for on-device agents. 24B total, 2B active per token. Liquid measured 184 tokens/s on an Apple M5 Max.
  • NVIDIA Nemotron 3.5 Bolt E4B: a compact model for local assistants, about 4B effective parameters.
  • Liquid AI LFM3: a family of small text, vision, audio and decision models, each picked by what it does and where it runs.

The "2B active" part is the trick. The model knows a lot (24B parameters), but each token only uses a small part of it, so it runs at a speed a laptop or a good phone can handle.

Where a local model wins

Not on raw quality. A frontier model in a data centre is still better at hard reasoning. A local model wins on everything around the answer:

1. Privacy by construction. The text never leaves the device. For a parking app, think of a driver's location history, or an owner's photos of their home driveway. "We don't send it" is a stronger promise than "we send it, but securely".

2. It works with no signal. I wrote about a trip where the signal dropped halfway through. Basement car parks are exactly where you lose network. A feature that needs the cloud fails at the moment it's needed.

3. No per-call cost. Once it's on the device, a thousand calls cost the same as one. That changes which features are worth building: things like checking every input or classifying every photo.

4. Latency without a network round trip. For small, frequent decisions this adds up quickly.

Where it doesn't

  • App size and memory. Even small models are hundreds of MB to a few GB. That's a big download for a parking app.
  • Battery and heat. Fine for occasional calls, harder for continuous ones.
  • Hard reasoning. Planning a multi-step task or writing long text is still better done remotely.
  • Updates. A model shipped inside an app updates when the app does.

The pattern I'd use: local first, remote when it matters

Use the small local model for the many small decisions, and the big remote model for the few hard ones:

Task Where
"Is this photo a parking spot?" Local
"Does this message mention a payment problem?" Local
Voice to text for search Local
Explain a disputed charge Remote
Plan a multi-stop trip Remote

The local model can also decide when to call the remote one, which is a nice use for the small decision models in LFM3.

What I'd tell you before you start

On-device AI isn't a smaller version of cloud AI. It's good at different things: private, offline, free per call, and instant. Pick the features where those matter, and keep the hard thinking in the cloud.

Sources: Liquid AI blog (Oct 13 and Oct 27, 2026) and NVIDIA model card (Oct 23, 2026), via "AI Model Releases: October 2026 Tracker", Digital Applied. Speed figures are the vendors' own.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.