Dev.to AI 🤖 Ai 👁 0 📖 2 min read

I rebuilt my local LLM tool engine so the model isn't the agent anymore

A while ago I posted the first version of this: a small engine that gives local models real tools. People used it, and I kept hitting the same problem. Small models are fine at writing. They're bad at being the agent. T

I rebuilt my local LLM tool engine so the model isn't the agent anymore

A while ago I posted the first version of this: a small engine that gives local models real tools. People used it, and I kept hitting the same problem.

Small models are fine at writing. They're bad at being the agent. They skip steps, guess the maths, lose track halfway through, and say "Done!" when nothing happened.

So in v2 I took that job away from the model.

L.A.T.E. in one paragraph

You plug in a model (Ollama, LM Studio, or any OpenAI-compatible API) and give it a goal. Plain code plans the steps. Tools do the exact work and check the result. The model only writes the words. It works the same whether the model is a 3B or something much bigger, as long as the rules can plan the job.

What it looks like

I ran these on qwen2.5:3b through Ollama.

One sentence, exact steps. The model doesn't plan this, code does:

![Multi-step job: two files merged into combined.txt]

It changes memory instead of piling up notes:

![Remember mango, change to apple, recall gives apple]

It tells you when a check fails. This is the part I care about most:

![A failing check ends with: NOT a success]

It checks its own work with tools. I corrupt a report on purpose, it finds the damage, repairs it, and verifies the repair:

![Corrupt, detect, repair, verify]

If a step fails you get "Partly done: 2 of 3 steps worked". If nothing ran you get "I didn't run any tools, so nothing was done". It never answers a job with "here's how you could do it".

What's new in v2

  • Rules plan the job, so the same words give the same plan every time.
  • Typed results: tools hand back real values (a number, a path, a count), so the next step doesn't have to guess from a sentence.
  • Verification by tools: compare numbers, compare text, "is this word in the file".
  • Memory that updates: "change my favorite fruit to apple" replaces the old note.
  • Faster: independent research steps run at the same time, and results are cached.
  • Small models aren't asked to write plans, because the rules do that.

The honest part

  • The rules understand shapes of requests, not all language. Unusual wording falls back to the model, and small models are shaky there. It can't fake success or hand you instructions, but it may do more or less work.
  • I ran four jobs on a real qwen2.5:3b for this post. The ~470 automated tests use fake models, including one that changes its mind on every call.
  • There's no delete, rename or overwrite tool. That's on purpose.
  • I built and audited this with a lot of AI help. Every bug I fixed has a test, but a test only covers what I thought of.

Try it

git clone https://github.com/ikaushik817-art/local-ai-tool-engine
cd local-ai-tool-engine
npm start

Node 18+ (see the README), MIT, two dependencies. Repo: https://github.com/ikaushik817-art/local-ai-tool-engine

If it does something wrong on your model, I'd love the exact prompt, what you expected, and what you got. That's the most useful thing you can send me.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.