WikiTrace: Why LLM Agents Shouldn't Blindly Trust Their Own Memory

When you give an LLM agent a memory system, you expect it to get smarter over time. It learns from past tasks, distills lessons, and applies them to new situations. But a growing body of evidence shows that memory can also make agents worse — causing them to fail on tasks they would have solved correctly without any memory at all.

Researchers from KAIST AI have a name for this: memory-induced degradation. And their new system, WikiTrace, is designed to fix it by giving agents the ability to verify their own memories before acting on them.

The Problem: Lessons Without Evidence

Standard agent memory works roughly like this: the agent runs a task, the trajectory (actions, observations, outcome) gets distilled into a concise lesson, and that lesson is stored for future use. When a new task arrives, the agent retrieves relevant lessons and applies them.

The flaw is subtle but consequential. Lessons are typically labeled only with whether the task succeeded or failed — not why. And they're handed to the agent without the original trajectory that produced them. So if a lesson is wrong, over-generalized, or misapplied to a different context, the agent has no way to catch the error. It just follows the rule.

The KAIST team found that over half of memory-induced degradation instances trace back to a single misleading memory item. The agent trusted a lesson it shouldn't have, and that trust cost it the task.

WikiTrace: Three Layers, One Descent Mechanism

WikiTrace restructures agent memory into three interconnected layers:

1. Insights — Concise, actionable guidelines: what to do, when it applies, and which past run it came from.
2. Compilations — A diagnostic summary explaining why that source run succeeded or failed.
3. Raw Trajectories — The unaltered execution log: every action, every observation, the final outcome.

Every insight is permanently linked downward to its compilation and raw trajectory. This is the key structural difference from prior systems like ExpeL or AWM, which store lessons but discard the evidence chain.

At inference time, the agent starts with all available insights in its prompt. But instead of blindly applying them, it's equipped with a descent mechanism: whenever an insight lacks necessary detail, conflicts with another insight, or originates from a failed trajectory, the agent uses terminal tools to traverse the evidence links — inspecting the compilation and raw trajectory before taking any environment action.

This is analogous to how a careful analyst doesn't just read a summary report; they go back to the source data when something doesn't add up.

What the Benchmarks Show

WikiTrace was evaluated across five agentic benchmark families spanning eight distinct evaluation suites: ALFWorld (household tasks), ScienceWorld (experiment simulation), PlanCraft (sequential crafting), τ³-bench (retail tool use), and AppWorld (complex API-driven tasks).

The results, reported in the paper, are measured along two axes: Score (task success rate, higher is better) and Regression (the rate at which memory turns prior successes into failures, lower is better).

The comparison between WikiTrace and "Insights Only" is particularly telling. Both systems receive the exact same distilled lessons. The only difference is that WikiTrace agents can descend into the evidence chain. That single capability accounts for a 3.2-point average score improvement and cuts the regression rate from 6.5% to 4.5%.

Degradation Scales With Task Difficulty

One of the more interesting findings is how the benefit of evidence descent scales with task complexity. The team analyzed descent frequency and performance delta across easy, medium, and hard tasks:

• Easy tasks: 22% descent rate → +0.8 score gain over Insights Only
• Medium tasks: 32% descent rate → +3.3 score gain
• Hard tasks: 44% descent rate → +12.4 score gain

On hard tasks, agents descend into raw trajectories nearly half the time — and the payoff is substantial. This makes intuitive sense: simple tasks have simple lessons that are hard to misapply. Complex tasks involve more edge cases, more context-dependence, and more opportunities for a distilled lesson to lead the agent astray.

Why This Matters for Agent System Design

The WikiTrace paper raises a design principle that's easy to overlook: memory without provenance is a liability. When you strip a lesson from its context and hand it to an agent as a rule, you're asking the agent to trust a summary it cannot verify. That works fine when the summary is accurate. It fails when the summary is wrong, incomplete, or applied to a situation it wasn't meant for.

This connects to a broader challenge in building reliable long-horizon agents. Systems like Reflexion and VOYAGER have shown that agents can improve through self-reflection and skill accumulation. WikiTrace adds a complementary capability: the ability to audit accumulated knowledge rather than just accumulate it.

The practical implication for anyone building agent systems is that the memory store shouldn't just hold lessons — it should hold the evidence those lessons were derived from, and the agent should have a principled way to consult that evidence when a lesson seems unreliable.

Implementation and Availability

The WikiTrace code is available on GitHub, and the paper is on arXiv. The system is model-agnostic in principle — the descent mechanism relies on terminal tool calls rather than any specific model architecture, so it can be layered onto existing agent frameworks.

The memory build phase (compiling insights and linking them to trajectories) happens offline, so it doesn't add latency to inference. The descent itself does consume additional context and tool calls, but the authors note that the memory corpus is mounted read-only and lookups don't count against the agent's environment step budget.

The Takeaway

Memory-induced degradation is a real and measurable problem in LLM agent systems. WikiTrace addresses it with a structurally simple idea: keep the evidence chain intact, and give the agent the tools to consult it. The benchmark results suggest this is worth doing — especially for complex, multi-step tasks where the cost of a wrong lesson is highest.

The broader lesson for agent system designers: trust, but verify. Even your own agent's memory.