Agent memory: where it lives, when it expires, how it is deleted

You deleted the memory file. The agent still remembers.

That is the first thing worth knowing about agent memory: the file you can see is rarely the thing the model reads. Behind it sits an index, and an index outlives its source unless something deletes both.

Most writing about agent memory argues about retrieval quality. The questions that decide whether you can operate the system are duller: which store does the memory live in, what writes to it, when does it expire, and what is actually gone after a delete.

Three things get called memory

A context window, a session log and a knowledge base share one word and almost nothing else.

The context window is assembled per turn and discarded after. A session log is append-only and exists to reconstruct what happened. A knowledge base is durable, indexed and queried. Conflating them is how retention questions go unanswered: you delete "the memory", and you have deleted one of three.

When it is written, and what rebuilds it

Saving the memory file rebuilds the index. In one agent workbench I maintain, the file API calls a rebuild when the path is the memory file, and the rebuild re-reads the file and re-embeds what it finds.

The interesting part is the condition. The guard hashes the file first, and the hash helper returns an empty string when the file does not exist:

    def _get_file_hash(self) -> str:
        """Get MD5 hash of MEMORY.md."""
        if not self._memory_path.exists():
            return ""
        content = self._memory_path.read_bytes()
        return hashlib.md5(content).hexdigest()

    def _maybe_rebuild(self) -> None:
        """Rebuild index if MEMORY.md has changed."""
        current_hash = self._get_file_hash()
        stored_hash = self._get_stored_hash()
        if current_hash and current_hash != stored_hash:
            self.rebuild_index()

Read the two branches together. Editing the file produces a different non-empty hash, so the index is rebuilt. Deleting the file produces an empty hash, so the rebuild never runs — and the previous index stays exactly where it was.

That single conditional is the difference between "the user deleted their memory" and "the user deleted the file we read memory from".

What the code says

The same question applies to spending. A memory system that queries on every turn pays for retrieval it already did. One companion app keeps a session snapshot with a TTL, a turn budget and a capacity cap:

export const DEFAULT_RECALL_TTL_MS = 15 * 60 * 1000;
export const DEFAULT_RECALL_MAX_TURNS = 10;
export const SNAPSHOT_MAX_MEMORIES = 40;

const RECALL_TRIGGER_PATTERN = /(记得|还记得|记不记得|上次|上回|之前|以前|曾经|我说过|跟你说过|那天|当时|早先)/;

Four constants decide how much of the conversation the model remembers for free. The trigger regex is written against Chinese phrases — 你还记得 (do you remember), 上次 (last time), 之前 (before) — because that is the language the product ships in. The lesson is not the language: when the user explicitly points at the past, the snapshot is bypassed and a real recall runs. Caching recall silently is how an assistant ends up saying "I don't have that" about something it was told an hour ago.

Deleting is a verification problem, not a delete call

The interesting part of deletion is not the call, it is the report. Delete functions usually count successes as "the gateway returned". That number is a claim about your code, not about the database.

The version I trust counts the state after deletion. It re-lists the source of the memory and reports how many rows are still visible:

  if (input.gateway.listBySourceConversation) {
    try {
      const stillThere = await input.gateway.listBySourceConversation({
        visitorId: input.visitorId,
        companionId: input.companionId,
        conversationId: input.conversationId,
        appId: input.appId,
      });
      result.remaining = stillThere.length;
    } catch (error) {
      result.failed += 1;
    }
  }

A delete that returned successfully but left the row behind counts as nothing. Because the whole operation is summed as scanned = skipped + deleted + failed, the arithmetic itself catches a candidate that was neither deleted nor reported as failed.

There is one more field worth copying: whether the candidate list was exhaustive. A search-based lister is truncated at some limit, and past that limit "not found" and "does not exist" are indistinguishable. If the backend cannot list by source, the report says partial instead of cleared.

When you should not have memory at all

Memory is a liability with a feature attached: it is personal data you now store, expire and delete on request. Three cases where the honest answer is no memory.

• The task is stateless. A formatter, a converter, a reviewer that takes a diff and returns comments: none of them benefit from remembering last week.
• You cannot name the deletion procedure. If the answer to "delete this user's memory" is "run the delete endpoint and hope", you have a compliance problem rather than a feature.
• The memory would only restate the recent conversation. Long-term memory earns its cost on cross-session, cross-time facts. Inside one session the transcript already has them.

The reverse view

The opposite design is defensible too: make memory explicit and user-visible, a file the person can open and edit, with the index treated as a cache that may be rebuilt at any time. That trades recall quality for inspectability, and for a personal tool it is often the better trade — the user can see what the agent knows and correct it in a text editor.

What does not work is the middle: an opaque store with a visible file in front of it, where the two can disagree and nobody can tell which one answered the question.

Both questions — where it lives, and what survives a delete — have to be answerable in one sentence each. If they are not, the memory design is not finished.

The full implementations are in the open-source repositories: Deep-Whisper for the snapshot and the deletion path, and Company-Brain for the knowledge-base side. The longer version of this article, with a file and line number on every claim, is at aiengstore.com/blog/agent-memory. More of these systems are at aiengstore.com.