My cheap models weren't dumb. They were missing a world model.

Last week I sat down to start on the self-evolution part of Mocu, the open-source agent app I've been building. I gave myself a vague goal: make the agents get better at their job without me hand-tuning prompts every time something breaks.

I spent the first few days doing what I always do — reading papers, sketching ideas on paper, then immediately poking holes in them. Every idea I had died within an hour of thinking about it. Then I remembered a paper I'd only skimmed months ago, about a self-evolving system that used planning to learn from its own mistakes. I went back to find it and ended up somewhere else entirely, at a paper whose title reminded me of two things I'd been chewing on for months:

Self-Evolving World Models for LLM Agent Planning (arXiv:2606.30639)

I've read maybe a third of it. I'm not going to pretend I've absorbed the whole thing. But it gave me a hypothesis I actually believe, and I'd rather write it down now, while it's still half-formed, than wait until I've either proven or abandoned it.

This is that write-up. Including the parts where I'm probably wrong.

The problem I keep running into

Mocu is designed to run on cheap models. That's the whole point — you bring your own OpenAI-compatible endpoint, and if you want to point it at a budget model or a local one, you should be able to, and the app should still be useful.

The trouble is that cheap models are noticeably worse at agent work. Not at chatting. At doing things: multi-step tasks, tool calls, recovery when something unexpected happens. A big model will fumble a step and quietly correct course. A small model will fumble a step and confidently keep going in the wrong direction for six more steps.

For a long time I assumed the answer was "bigger models have seen more patterns, so they generalize to more situations." That's true, but it's also useless as a design principle, because it just means "pay more." I looked at building a custom world model for my agents a while back. The data requirement and the cost killed it before I wrote a line of code.

The ideas that died

For the record, so this doesn't look like I had a clean insight:

• Fine-tune a small model on successful agent traces. Cost, data volume, and it goes stale the moment you change a tool.
• Give the agent a big library of past failures in the prompt. Context bloat. The model reads twenty failures and picks the wrong lesson.
• Add a critic model that reviews each step. Doubles the cost, and the critic is as dumb as the actor, so it approves bad steps.
• Just write better prompts. I've been doing this for months. It's how I got here.

Each one failed the same test: I could imagine it working in a demo and failing in week three.

The paper, briefly

The paper proposes something called WorldEvolver: a world model that improves without touching model weights. The agent stays frozen. The world model gets better by updating its context as it goes.

Three pieces did the work for me:

1. Episodic memory. It stores real action→outcome transitions and retrieves similar past ones when it needs to predict what a new action will do. Not a summary. Actual transitions.
2. Semantic memory. When a prediction doesn't match what happened, it converts that mismatch into a written rule — a persistent heuristic — so the same mistake doesn't get predicted again. Again, no gradient descent. Just text.
3. Selective foresight. It throws away low-confidence predictions before feeding them into the agent's reasoning. This one is the detail I keep thinking about.

The authors report better prediction accuracy across three backbones and improved downstream success on ALFWorld and ScienceWorld. It was accepted at EMNLP 2026 Findings. I'm taking those numbers at face value for now; I haven't tried to reproduce anything.

The hypothesis I actually believe

Here's the part that isn't in the paper, and the part I'm least sure about.

I think the "reasoning" that big models do — the chain of thought — is a world model. Or close enough that the distinction doesn't matter for my purposes.

When a big model thinks "if I run this command, the file gets overwritten, so I should back it up first," it's doing exactly what a world model does: given an action, estimate the outcome. It's just doing it in prose, inside one forward pass, without a name for it. A dedicated world model does the same thing more explicitly, with memory, and with a confidence score attached.

If that's roughly right, then the gap between cheap and expensive models might not be "knowledge" or "pattern count" so much as simulated consequences per step. Big models run more of them, more reliably. Small models skip the step where they imagine what happens next.

And that's a gap you can close from the outside — without retraining, without a bigger model. You give the small model the foresight it won't generate on its own.

What I'm building

The loop I want in Mocu looks roughly like this:

before an action:
  predict outcome + confidence   (from episodic memory + current state)

if confidence is low:
  drop the prediction, act anyway

after the action:
  observe real outcome
  if prediction != observation:
      write a rule to semantic memory
      store the transition in episodic memory

The agent's parameters never change. The world model's context evolves. That's what "self-evolving" means here, and it's the reason it's cheap: no training run, no dataset, no GPU hours. You pay for a few extra tokens per step and you get a system that accumulates judgment.

Two things I care about that the paper doesn't have to care about:

It has to be inspectable. Mocu's users should be able to read the rules the world model has learned. If the system concludes "when the user asks to deploy, always check the build directory first," that should be a sentence in a file, not a vector in a matrix. Semantic memory as text is the whole reason this fits an open-source desktop app.

It has to be honest about uncertainty. The confidence filter matters more than the prediction. A small model with a confidently wrong world model is worse than a small model with no world model at all. I'd rather the system say "no idea" and act than invent a plausible outcome and commit to it.

Why this matters for Mocu specifically

Mocu is a desktop app, Apache 2.0, running on your machine with your own model keys. Everything — agents, skills, projects, memory — lives in local files you can read and edit. There's no server to upload your traces to and no training pipeline to rent.

Which means the only kind of self-improvement that's actually available to me is the kind that lives in text: rules, transitions, notes. That's a constraint I didn't choose, but it happens to be the same constraint the paper works within. Maybe that's luck. Maybe it's why the idea landed so hard.

If it works, the payoff is specific: a user running Mocu on a cheap model gets an agent that has read its own failure history, writes down what it learned, and stops repeating itself. That's a different product from "pay more for a smarter model."

What would make me abandon this

I want to write this down so I can't quietly move the goalposts later.

• Confidence scores aren't calibrated. The paper leans on the model's self-reported confidence, and on a lot of cheap endpoints that number is either missing or fiction. I may end up deriving confidence from agreement across a few sampled predictions instead. That costs more tokens.
• Rules accumulate and contradict. Write enough heuristics from single observations and you get a context window full of superstition. The fix is probably requiring a rule to recur before it's promoted — which means the system learns slower, and might learn nothing at all in a session.
• It doubles the calls. One prediction per action is not free. If the wins don't beat the cost, this is just an expensive way to be wrong.
• Self-confirming memory. If the agent retrieves its own past predictions instead of past outcomes, it will happily agree with itself forever. The episodic store has to be outcomes only.
• The paper tested text environments. ALFWorld and ScienceWorld. Not the web, not code execution, not whatever messy thing your agent actually does. I'm not assuming the result transfers.

Any one of these could kill it. The third one probably should kill it if I'm honest about the economics.

What I'll report back

I'm going to run this on a real budget model against a set of tasks that currently fail, and I'll publish the numbers whether they're good or embarrassing:

• task completion rate, before and after
• steps to completion (cheap models fail by wandering)
• cost per completed task, including the prediction calls
• and, qualitatively, examples of rules the system wrote — because I think that's the interesting artifact, not the benchmark

If it doesn't work, that's also worth a post. The reason I'm writing this now, before the results, is that a negative result is only interesting if the prediction was on the record first.

If you're building the same thing

Mocu is open source — it's an agent app that runs on whatever OpenAI-compatible endpoint you point it at, including small local ones: github.com/kharchangai/Mocu.

If you've already tried something like this — externalized prediction, memory that writes its own rules, agents that are allowed to be unsure — I'd genuinely like to hear what broke. Especially the parts where the memory poisoned itself. That's the failure I'm most worried about.

And if you think the whole premise is wrong, that's useful too. Tell me which of the five failure modes above gets me first.