Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 14 min read

Memory Curation - Solving the Problem of Forgetful Agents.

Technical Note One of the more frustrating problems in running an agentic engineering organization is explaining a decision, seeing the agent acknowledge it, and then watching it follow an older instruction a few sessio

Technical Note

One of the more frustrating problems in running an agentic engineering organization is explaining a decision, seeing the agent acknowledge it, and then watching it follow an older instruction a few sessions later. You look at its memory and find both the original instruction and the decision that replaced it. The agent is working from the wrong one.

At InflectAI, we had accumulated roughly 1,400 linked memory files. Agents had their own memory, a shared index, session summaries, and handoffs for the next time they resumed. We had asked them to preserve standing instructions in those handoffs. Each agent restated the instructions in its own words. When a decision changed, we updated some of those copies. The rest remained available to be loaded after compaction, when a long conversation is reduced to a summary and the agent resumes with reconstructed context.1

We also found memory that never reached the agent at all. In our observed setup, a hook output above roughly 10,000 characters could arrive as a preview of about 2 KB. A handoff could exist in full on disk while most of it was absent from the context the agent received. Later, we found current rulings surviving only as titles because older material had consumed the space for their actual instructions.

We kept adding memory to preserve continuity. That also kept the old instructions available to be loaded again. We needed a way to retire them and make sure the current instruction reached the agent.

1. Trace what the agent actually receives

Before changing the model or buying a larger context window, inspect the inputs to one resumed session. We had five places to check: the top-level instructions, shared memory index, agent-specific memory, handoff, and automatic compaction summary. A rule could appear in several of them, with slightly different wording and no indication that it had ended.

For one instruction that the agent got wrong, collect these fields from every source:

Field Question
Source Where did this wording come from?
Authority Was it the operator's instruction, an agent's interpretation, or historical discussion?
Status Is it current, replaced, or unresolved?
Delivery Did the agent receive the body, a title, a preview, or nothing?
Ordering What consumed the context budget before this instruction?

Read the rendered hook output as well as the files. An instruction can exist on disk and appear by name in the startup index without its actual wording ever reaching the agent.

We found two failures that looked similar from the outside. In one, the agent received an obsolete instruction and treated it as current. In the other, the current instruction was cut from the material it received. Putting another reminder in the handoff could make both harder to diagnose.

2. Give current instructions one maintained home

We created a small, separately versioned store for operational rulings. One topic has one maintained file. The original words carry a source reference. The memory keeper's plain reading is labeled separately. When they differ, the operator's words govern.

Here is a synthetic record showing the shape. These are example instructions, not our deployment policy:

TOPIC: Production deployment review
APPLIES TO: release operators

## LIVE
Operator decision example-002:
"Review the exact artifact that will be deployed."
Plain reading (memory keeper): An older artifact's review does not transfer.

## REPLACES
Decision example-001: the earlier deployment procedure.

## HISTORY
Decision example-001, ENDED when example-002 took effect.

The historical record stays. It stops being part of the current-instruction payload. Handoffs keep the work in flight, the agent's own decisions, and pointers to the rulings it needs. They stop paraphrasing standing instructions into another copy that someone will eventually have to reconcile.

Our live_section parser selects the active section before the renderer builds the payload:

m = re.search(r"^## LIVE\n(.*?)(?=^## |\Z)", text, re.S | re.M)
body = m.group(1).strip() if m else ""

The parser selects the LIVE section. History remains available for an audit without being presented as a competing instruction. The Markdown format is our convention. In the runnable example, the same distinction is a status field in JSON. The operator controls this store; an agent's memory or a tool result cannot promote itself into a new ruling. A source ID helps trace authority. It doesn't authenticate itself.

Retirement is a separate job. Updating the canonical file doesn't remove the old sentence from an existing handoff. We record the wording being replaced, search for surviving copies, and retire or annotate them with a pointer to the current source. Our system runs a literal scan every six hours and a model-assisted review of changed files daily. The scan catches known old wording. The model read can find paraphrases; its findings still need review.

3. Curate the context budget

Once we had a current source, we still had to decide how much of it each agent received. Loading everything in alphabetical order let older topics fill the budget before a newly changed instruction. A new rule could appear in the index while its body was omitted.

Our startup renderer now ranks instructions for the particular agent, then its area, then fleet-wide instructions. Explicit pins can put selected instructions first. Within a rank, recently changed files receive priority. This is the exact ranking excerpt read from the implementation on November 17:

try:
    age = time.time() - os.path.getmtime(os.path.join(lib.RULINGS, f))
except OSError:
    age = 1e9
ranked.append((rank, age if age < 3 * 86400 else 1e9, t, sha))
ranked.sort()

File modification time is a scheduling shortcut here. The source record establishes which instruction is authoritative; a recently touched file doesn't acquire authority just by being recent. If you restore or copy the store, modification times can change. An explicit effective-time field would make that priority easier to reproduce.

We also budget the actual text emitted. In the November 13 architecture, agent memory has a 9,500-character budget and handoffs have 9,000. Those are local implementation settings derived from the truncation we observed, not universal Claude Code limits. Headers, source references, and omission notices need space too.

A rule that doesn't fit needs an explicit disposition. Can it be summarized with a source pointer? Must another rule move down? Should this action stop until the rule can be retrieved? Silently dropping the tail makes the agent's behavior difficult to explain later.

The companion example keeps that bookkeeping visible:

rest = [name for name in remaining if name != record["id"]]
tail = "\nNOT SHOWN: " + ", ".join(rest) if rest else ""
candidate = HEADER + "\n".join(blocks + [block]) + tail
if len(candidate) > cap:
    continue
blocks.append(block)
remaining = rest
next_seen[record["id"]] = {"digest": digest, "offset": offset}

This is a teaching adaptation, not a production excerpt. It only marks a record emitted after its complete block fits. An omitted rule stays eligible for delivery on the next relevant call. If even the omission notice exceeds the budget, the example raises an error rather than returning a payload that it knows is too large. Its cap measures the context string. Leave headroom for the JSON envelope and check the complete hook output against your runtime's limits.

Keep an over-budget rule out of the delivery record. If you mark it as shown before rendering, the next call can suppress an instruction the agent never received. Test that the omitted rule is still eligible when more room becomes available.2

4. Rebuild context after compaction, then refresh it during work

A session's old delivery record shouldn't suppress instructions after its context has been compacted. Our startup hook rebuilds the applicable current instructions and replaces the record of what has been shown. The adapter returns context in the hook envelope:

print(json.dumps({"hookSpecificOutput": {
    "hookEventName": "SessionStart",
    "additionalContext": text
}}))

The hook's delivery record tells us what we emitted. It doesn't establish that the agent read it or will follow it.

During the session, the act lookup selects the rules relevant to the proposed tool call. Module edits also bring in a small design card identifying the component's responsibilities, sources, and implemented guards. The cards are checked for stale citations. We don't copy the entire architecture document into every session.

The implementation uses content fingerprints to notice changes and transcript growth to decide when a card should be shown again. This exact excerpt is from the card-injection hook:

sha = hashlib.sha1(card["text"].encode()).hexdigest()[:12]
prev = shown.get(card["path"])
if prev and prev[0] == sha and now - prev[1] < reshow:
    continue

Here now is the transcript file's byte length. The configured distance was 400,000 bytes, described internally as roughly 50,000 tokens. It is a heuristic. Transcript serialization, tool output, and language all affect the relationship between bytes and model context. This hash is a change detector, not a signature proving who authorized the card.

Track subagents separately. Our act hook combines the parent session ID with the subagent ID. Otherwise a parent seeing a rule can suppress delivery to an executor that never received it. Compaction, transcript replacement, and child-agent creation all need their own tests.

5. Keep reminders separate from refusals

We found that agents could skim injected instructions under load. The next step was to enforce the rules that a machine could check. We retain text for judgment calls and use guards for explicit conditions such as a required review record being absent.

There was another copying failure here. A script had its own guard and continued using an old threshold for roughly 30 hours after the ruling changed. The new instruction existed. The script still ran the old comparison. We needed one implementation of the guard as well as one current record of the rule.

A context hook and a guard have different outputs. This is the documented Claude Code shape for refusing a proposed tool call:

{
  "hookSpecificOutput": {
    "hookEventName": "PreToolUse",
    "permissionDecision": "deny",
    "permissionDecisionReason": "The required review for this artifact is missing."
  }
}

Use the hook reference for the runtime you deploy. SessionStart can supply context; PreToolUse can return a denial. Supplying additionalContext alone does not refuse the call. A one-time denial followed by an automatic pass is a reading interruption, not a check that the action now satisfies the rule.

The reference example deliberately supplies context only. A production write should have its authorization checks inside the trusted execution path as well. If an agent can call another tool or modify the hook, a local reminder is a weak boundary. In our architecture, advisory delivery can fail open; the production-operation invariant guard is the explicitly fail-closed exception. Choose that behavior per operation and test missing files, exceptions, timeouts, and malformed input.

6. Worked example: the old instruction loads and the new one disappears

Now take one instruction through both loaders. The instructions and the 500-character budget below are synthetic. Both instructions exist in storage. We can print exactly what reaches the agent and see where the replacement disappears.

The old instruction allows deployment after the branch is reviewed. The replacement requires review of the exact artifact. Between them sit investigation notes from the previous session. An append-only loader concatenates the material and keeps the first 500 characters:

BUDGET = 500
OLD = "Deploy after the branch is reviewed."
NEW = "Deploy only after the exact artifact is reviewed."
NOTES = "Investigation notes from the previous session. " * 20

combined = "\n\n".join([OLD, NOTES, NEW])
legacy = combined[:BUDGET]

The new instruction made it into storage. It came after the material that filled the budget. These are the results from the runnable worked_example.py:

APPEND-ONLY LOADER
Old instruction loaded: True
New instruction exists in the input: True
New instruction loaded: False
Delivered characters: 500

Now curate the input. Mark the old instruction ended and give the current instruction priority over the notes. plan_context filters out ended records, fits complete blocks within the budget, and names what it couldn't include. It is defined in memory_context.py.

from memory_context import plan_context

def record(name, text, status, priority, version=1):
    return {
        "id": name, "text": text, "status": status, "priority": priority,
        "version": version, "applies_to": ["*"], "acts": ["deployment"],
        "source": "operator-decision:example-" + str(version),
    }

records = [
    record("old-deploy", OLD, "ENDED", 0),
    record("notes", NOTES, "LIVE", 10),
    record("deploy", NEW, "LIVE", 0, version=2),
]
ordered = sorted(records, key=lambda r: r["priority"])
curated, seen = plan_context(ordered, None, {"deployment"}, {}, cap=BUDGET)
print(curated)

The same budget now delivers the current instruction and its source. It also says the notes were omitted:

CURRENT OPERATOR RULES: these replace conflicting historical summaries.
Delivery records context emitted, not understanding or authorization.
[deploy v2] Deploy only after the exact artifact is reviewed.
Source: operator-decision:example-2

NOT SHOWN: notes

The old instruction stays in the historical record, where we can inspect it later. The loader leaves it out of the active payload and puts the current instruction ahead of the notes. If the notes don't fit, it tells us.

Change the instruction once more in the same session. Keep version 2 as history, update the live record, and pass the previous delivery state back to the loader:

current = records[2]
records.append(dict(current, id="deploy-v2-history", status="ENDED"))
current.update(version=3, source="operator-decision:example-3",
               text=NEW + " Record the rollback plan too.")
ordered = sorted(records, key=lambda r: r["priority"])
refreshed, seen = plan_context(ordered, None, {"deployment"}, seen, cap=BUDGET)
print(refreshed)
CURRENT OPERATOR RULES: these replace conflicting historical summaries.
Delivery records context emitted, not understanding or authorization.
[deploy v3] Deploy only after the exact artifact is reviewed. Record the rollback plan too.
Source: operator-decision:example-3

NOT SHOWN: notes

The instruction is delivered again because its fingerprint changed. The loader doesn't require the agent to remember that it should look for an update. To reproduce all three outputs, put worked_example.py and memory_context.py in the same directory and run:

python3 worked_example.py

The same budget now carries the current instruction, and a later update is delivered in the same session. This replay checks delivery; it makes no model calls.

7. Run the example against failures you expect to see

The companion has four files: worked_example.py, memory_context.py, rules.json, and test_memory_context.py. It uses the Python standard library and makes no network calls. It selects live records, filters by action, applies a character budget, fingerprints changes, resets delivery state at startup, and refreshes context after transcript growth. It matches a literal example MCP tool name. It doesn't parse arbitrary shell commands.

This example concerns runtime operator-instruction selection and context delivery. It is not a model of cognitive Working Memory or of persistent semantic, episodic, or procedural memory.

Place the four files together and run:

python3 -m unittest -v test_memory_context.py
printf '%s\n' '{"hook_event_name":"SessionStart","session_id":"demo"}' \
  | python3 memory_context.py

The second command creates local demonstration state under .example-state/. It doesn't install a hook or change any running agent. To try the adapter, merge these entries into a disposable project's existing hook configuration, replacing the path with your absolute example path:

{
  "hooks": {
    "SessionStart": [{
      "matcher": "startup|resume|clear|compact",
      "hooks": [{"type": "command", "command": "python3 /absolute/path/memory_context.py"}]
    }],
    "PreToolUse": [{
      "matcher": "mcp__example__deploy",
      "hooks": [{"type": "command", "command": "python3 /absolute/path/memory_context.py"}]
    }]
  }
}

The MCP name is a fixture; this does not create a deployment tool. These entries show the adapter contract, not a live installation tested against every client version. The example treats identity as unknown and widens advisory context. A real adapter should get agent identity from an operator-controlled registry, never from an identity the model asserts in a tool argument. Serialize calls per session or replace its small JSON state file with transactional storage.

The tests cover twelve cases:

Case Expected result
First relevant action Current instruction appears with its source.
Retired instruction Historical text stays out of the payload.
Unchanged instruction Repeat delivery is suppressed within the configured distance.
Text changes without a version bump The fingerprint changes and context is reissued.
Compaction reset An empty delivery record makes current context eligible again.
Transcript growth The same content is refreshed at the configured boundary.
Transcript shrinks Refresh rather than treating a negative distance as recent delivery.
Known versus unknown actor Scope narrows for a known actor; unknown identity grants no permission.
Rule exceeds budget Report the omission and leave it unmarked as emitted.
Large rule precedes a small one The smaller rule can still fit.
Unrelated action Nothing is injected.
Session startup All applicable live records are eligible for delivery.

Three regression tests check the worked example: the append-only loader drops the replacement, the curated loader supplies it within the same budget, and a subsequent operator change refreshes the instruction in the same session. Four subprocess tests exercise the JSON adapter: context output without a permission grant, repeat suppression followed by compaction reset, parent/child separation, and an unrelated event remaining silent. All 19 tests run without calling a model. In a runtime pilot, inspect the text the agent actually receives, a replaced instruction after resume, and a known-bad action reaching the real execution tool. A drill that read no records should report missing evidence, not a pass.

What changed for us

We started by trying to preserve what the agents knew. We learned that preserving every copy also preserved decisions we had already changed. Then we found instructions on disk that weren't reaching the agents at all. Curating the memory meant deciding which source governed, retiring old wording, and making the delivery process show us what it left out.

We call the resulting system the Sabriel Memory Control Plane, after our memory-keeper agent. It now also supports design context, action guards, task delivery, and audits. If you're building something similar, start with a decision you've already changed. Find the old copies, mark which instruction is current, and inspect what the next session actually receives. Then change the instruction again and check that the next session gets the update.

  1. Internal architecture and recorded failure history, November 6-13, 2026. File counts and the output-preview behavior describe the environment we inspected, not general product limits. The November 13 document reports 77 ruling topics and 161 recognized action categories. Those are scope counts, not efficacy measurements. ↩

  2. The internal architecture reports a replay in which all 1,011 operations recognized by the existing invariant guard mapped to an action category among 27,597 tool calls. That tests agreement with the existing recognizer. It does not establish coverage of every consequential operation or measure compliance. During source inspection on November 17, we found a hook marking instructions as delivered before their bodies fit the output budget. The repair is merged and installed. We checked the installed files against the merged source and reran 24 action-hook tests and six startup cases successfully, including an omitted instruction being delivered on the next matching call. The installed renderer can deliver an oversized instruction in short form with a source pointer; an instruction only named as omitted remains eligible for delivery. These checks used test fixtures on the installed code. We did not observe a live agent session receiving the deferred instruction or establish that the agent followed it. ↩

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.