Backprop for Systems, Not Models

Your agent system does not learn. It just runs.

Most multi-agent systems I see have the same lifecycle. They run, they fail, a human reads the logs, and the human edits a prompt. Then they run again.

We call that iteration. It is closer to a neural network where a person adjusts each weight by hand after every wrong answer. Nobody would accept that for a model. We accept it for the system around the model because we have no name for the missing part.

The missing part is a backward pass. I am not talking about fine-tuning. The model stays frozen. I mean a mechanism that takes a failure at the output and pushes blame back through the workflow, so the prompts, the routing, and the memory change in response.

I have spent six years building forward passes. This article is my attempt to think about the other direction, and to be honest about where the idea is weaker than it sounds.

What backpropagation actually needs

Strip away the calculus and backpropagation needs four things.

1. A forward pass that records its intermediate states.
2. A loss that says how wrong the output was.
3. A rule for assigning blame to every component that contributed.
4. Parameters that can be adjusted by a controlled amount.

In a neural network all four come for free. The activations are stored, the loss is a number, the chain rule assigns blame, and the learning rate controls the step.

In an LLM workflow none of them come for free. Intermediate states are thrown away or flattened into a log line. The loss is a vague feeling that the answer was bad. Blame is whoever the engineer suspects first. The parameters are prompts and config files that get rewritten wholesale.

So the question is not whether we can copy the algorithm. We cannot, nothing here is differentiable. The question is whether we can rebuild each of the four requirements out of the materials an agent system actually has.

The forward pass I already built

Looking back, most of my independent work was about making the forward pass visible. I did not plan it that way.

At Abzu, which I co-founded in 2017 (alongside people way smarter then me), we worked on the QLattice, a symbolic regression primitive. Symbolic regression is search over expressions guided by how well they fit. In 2020 I built Evos, genetic programming over executable symbol closures. Both taught me the same thing: you can improve a structure without gradients, as long as you can measure fitness and keep what works.

In 2021 I built Ant-Sim, a simulator of task allocation in the style of Deborah Gordon's harvester ants. No ant knows the plan. Each one changes task based on how often it meets other ants doing something. The colony adapts with no central signal at all. Keep that in mind, it comes back later as the main objection to this whole article.

In 2024 I released OrKa-reasoning, an open-source orchestration layer where every agent has a role, an input and output contract, and a trace. The point was never orchestration for its own sake. The point was that reasoning should be observable step by step.

In late 2025 I wrote about two orchestration loops: a linear one for step-by-step analysis, and a circular one for live conversation, where background agents enrich context around a central responder. I have since been thinking of that circular loop as a room: a user, a speaker, and a set of agents each dedicated to one task.

In August 2026 I published "AI Memory Should Not Live Inside the Model". The prototype behind it is a memory hub: plain Markdown memories served over HTTP, reached per session through MCP, each one carrying provenance, a trust state, and a staleness flag.

Put these side by side and you have recorded activations (the traces), a recurrent architecture (the room), and a parameter store that lives outside the model (the memory). Three of the four requirements are half-built. What none of it does is learn from its own failures.

The backward pass

Start with what kills it before it begins. A pattern I keep running into in pollers and fetchers is the boolean status: a source returned nothing, so the step is marked "skipped". That one flag hides four different facts. There was no new content, or the source was unreachable, or the response was malformed, or the fetcher hit a capability limit.

I call this "empty is not a state". In gradient terms it is a vanishing gradient. The information needed to assign blame was destroyed on the way forward, so no backward signal can ever tell the causes apart. A system that wants to learn has to keep typed reasons and attempt history as first-class records. That is the price of entry.

With that in place, the backward pass looks like this.

1. Trace the run. Every agent call stores what it received, what it returned, which memories it read, and which route it took.
2. Wait for a loss event. A human correction, a claim later proven wrong, the same wrong path taken twice. These arrive late and rarely.
3. Walk the trace in reverse. At each node, ask one question: given what this node received, was its output reasonable? If not, what in its inputs or configuration made it wrong?
4. Collect the answers as blame. Each answer is a short critique attached to a specific parameter: this prompt, this routing rule, this memory entry.
5. Aggregate before acting. One failure is an anecdote. Critiques are grouped per parameter across many runs.
6. Propose a small, versioned update. Rewrite one prompt, change one route, lower the trust of one memory. Every change can be rolled back.

The parameters here are not weights. They are prompts, routing rules, the assignment of agents to tasks, retrieval settings, and memory entries with their trust states.

The last one is where I think the real value is. When an output is proven wrong, blame should flow back to the memories that informed it and lower their trust. Trust stops being a label someone assigned at write time. It becomes a weight learned from outcomes. A memory that keeps leading to corrected answers decays. One that keeps surviving contact with reality hardens.

I should be clear about what is not new. Textual gradients over prompts exist already. TextGrad, the DSPy optimizers, and Microsoft's Trace all push natural-language feedback backward through a pipeline. They mostly work on fixed graphs, scored on benchmarks, with prompts as the only parameters. What I have not seen done well is the same idea over a circular loop with a human inside it, shared memory as the parameter set, and a loss that accumulates over months instead of minutes.

The judge is the weakest part

Step 3 above hides the biggest risk. Something has to look at a node and decide whether it was at fault. The obvious candidate is another LLM.

I have been burned by this already. When I rebuilt the OrKa benchmark, I found that my LLM judge had a position bias of 74.4%. In pairwise comparisons it favored an answer largely because of where it sat, not what it said. That invalidated my first round of results. The second version only stands because every comparison is counterbalanced.

Now imagine that judge computing gradients. Its bias does not stay in a score table. It gets written into every prompt and every memory it touches. The system optimizes toward the critic's taste, and every local metric keeps looking fine while it drifts.

In a neural network a bad loss function gives you a bad model. Here a bad critic gives you a system that is confidently teaching itself the wrong lesson. So I would hold three rules.

• The loss must come from the world, not from a model. Human corrections, facts later verified or disproven, repeated failures.
• The critic that assigns blame gets the same treatment as a benchmark judge: counterbalanced order, repeated runs, and a measured bias before it is trusted.
• Where it is affordable, replace opinion with replay. Rerun the failed episode with one component swapped and see if the failure disappears. That is a finite-difference gradient. It is expensive, and it is the only blame signal that is not another model's opinion.

Where the metaphor breaks

I like the backprop framing because it forces the right questions. I do not trust it past that point, for two reasons.

The first is smoothness. A gradient step is small and its effect is predictable nearby. A prompt rewrite is neither. Changing one sentence can fix the failing case and break ten cases nobody was watching. There is no learning rate for text. The closest thing is discipline: small edits, a frozen set of past episodes replayed before any change lands, and a rule that updates come from batches and never from a single bad run.

The second is time. A room is a recurrent system, so blame has to travel through turns as well as through agents. That is backpropagation through time, and it has the same problem it has in recurrent networks, only worse. Asking a chain of critics to blame turn 3 for a failure at turn 40 produces confident nonsense. The backward walk has to be truncated to a few turns, and anything older has to be handled by a different mechanism.

So "backprop" here is a name for a discipline of credit assignment. It is not an algorithm with guarantees. Anyone selling it as one is selling hype.

Ants do not backprop

Here is the objection I cannot get around, and it comes from my own simulator.

Backpropagation is a centralized algorithm. It needs one owner of the loss and one owner of the graph. Somebody has to hold the whole picture and walk it backward.

An ant colony has neither. In Ant-Sim, each ant adjusted its behavior from local encounter rates. There was no global error and no backward walk, and the colony still shifted workers toward what needed doing. The learning lived in the local rules.

For about ten years my work has circled one question: how does a system stabilize when no one owns the source of truth? Backprop answers it by appointing an owner. That is a legitimate answer for a small pipeline. For a room with many agents, shared memory, and several humans who disagree, I think it is the wrong default.

So the design I believe in is a hybrid.

• Local rules do most of the work. Each agent adjusts its own thresholds and its trust in the memories it reads, using only the signals it meets directly. A memory that gets contradicted loses trust at the point of contradiction, with no central pass.
• Global sweeps are rare. A full backward walk is triggered only by a costly failure, the kind where a human had to step in and undo something.

This is not exotic. Research on alternatives to backprop in neural networks keeps arriving at a similar shape: local learning rules, plus a sparse global signal that modulates them. The colony version of that is what I want to build.

What I am going to test

An idea like this is worth nothing until it survives a measurement, so here is the experiment.

The memory hub already has a long-running comparison planned: memory on versus memory off, on matched tasks, over roughly six months. The measures are human corrections, repeated wrong paths, and wall-clock time. Token counts are secondary. I would extend it to three arms.

Three results would prove me wrong.

• The local arm matches the sweep arm. Then the backward pass is ceremony and the ants were right all along.
• The sweep arm improves the critic's scores but human corrections do not drop. Then the system learned the judge, not the task.
• Either learning arm gets worse on the frozen replay set over time. Then the updates are drift, and hand editing was the safer process.

I do not know which way it will go. I have a guess that most of the gain comes from the cheapest part, trust that decays when reality contradicts a memory. If so, the honest title of the follow-up will be less exciting than this one.

The model does not need to change for the system to learn. But the system only learns if it keeps what it did, measures against the world, and is suspicious of its own judge.

Earlier work referenced

• OrKa-reasoning, open-source orchestration framework (PyPI)
• OrKa Brain paper, Zenodo, DOI 10.5281/zenodo.19227514
• How to Design Two Practical Orchestration Loops for LLM Agents, dev.to
• "AI Memory Should Not Live Inside the Model", dev.to, August 2026
• Ant-Sim (2021) and Evos (2020), earlier simulators and experiments