How to Implement Long-Term Memory for AI Companions: Pinning, Compaction, and Lessons from Mem0 and Letta

A user tells their AI companion on Monday that they are allergic to peanuts. On Thursday they ask for a snack idea. The model suggests peanut butter cookies.

Nothing crashed. The fact was stored somewhere. It just was not in front of the model when it mattered.

That is the core problem of long-term memory for companion apps and personal agents. It is not really a storage problem. It is a question of what the model sees on each turn, and what happens to everything else when the conversation outgrows the context window.

I work on Sipplay, an AI character chat and interactive story product. Our users hold conversations that run for weeks, so we have spent a lot of time on this. In this post I will walk through three questions every memory system has to answer, show how two of the strongest open-source projects answer them, and explain the choices we made.

Three questions every memory system answers

However the architecture diagram looks, a memory system for a conversational agent makes three decisions:

1. What is always in context? Some facts are too important to leave to retrieval: who the user is, hard constraints, the current relationship or task.
2. What happens at compaction? When the conversation is too long, something gets summarized, truncated, or moved out. The question is what survives that step.
3. What gets retrieved, and when? Everything not in context must be found again. That means deciding when to search, what to search for, and which of several similar records is still true.

Most "my AI forgot" bugs come from one of these three, and they need different fixes. A bigger context window helps with none of them for long: it only postpones question 2.

Mem0: a memory layer you bolt onto any agent

Mem0 (around 68,000 GitHub stars at the time of writing, Apache-2.0) is the most popular open-source memory layer. It sits beside your agent rather than inside it. The basic loop from its README is two calls:

from mem0 import Memory

memory = Memory()

# before replying: fetch what is relevant to this message
relevant = memory.search(query=message, filters={"user_id": user_id}, top_k=3)

# after replying: let Mem0 extract memories from the exchange
memory.add(messages, user_id=user_id)

Mem0 keeps memory at user, session, and agent level, so the same store can hold "Alice prefers dark mode" and "this support session is about billing".

The interesting part is how its extraction changed. In its April 2026 algorithm update, Mem0 moved to single-pass, ADD-only extraction: one LLM call per exchange, no UPDATE or DELETE operations. The README puts it plainly: "Memories accumulate; nothing is overwritten." Conflicts are handled at read time instead, with:

• Multi-signal retrieval: semantic similarity, BM25 keyword matching, and entity matching scored in parallel and fused.
• Entity linking: entities are extracted and linked across memories to boost retrieval.
• Temporal reasoning: time-aware ranking that picks the right dated instance for questions about current state, past events, or upcoming plans.

Mem0 reports 92.5 on LoCoMo and 94.4 on LongMemEval with this approach, at about 7,000 tokens per query. Note the caveat in their own README: those scores come from Mem0's managed platform, which includes optimizations not in the open-source SDK.

What Mem0 gets right: it treats memory as an append-only log plus a smart reader. Writes are cheap and never destroy information. "The meeting moved from Thursday to Friday" becomes two dated facts, and the reader's job is to pick the current one.

What to watch: nothing is always-in-context by default. If a fact must never be missed, like an allergy, you are relying on retrieval to surface it every time it matters. And the quality of "pick the current one" now depends entirely on ranking.

Letta: an agent that manages its own context

Letta started life as MemGPT, the research project that framed LLM memory as an operating-system problem: a small, fast main context and a large external store, with the model paging information between them. The original repository (about 25,000 stars) is now archived as V1. Active development has moved to letta-code, which Letta calls an agent harness.

Letta answers the three questions very differently from Mem0. Memory belongs to the agent, and the agent edits it itself.

Always in context: memory lives as Markdown files in a git-versioned memory filesystem (MemFS). According to the Letta docs, files under system/ are in the system prompt on every turn. Everything else stays out of context: the agent sees the file tree and reads what it needs.

Compaction: Letta has a background process it calls "dreaming". Subagents review recent conversations, consolidate lessons, and update memory without interrupting the active session. Crucially, you can trigger it on compaction:

const agentId = await client.createAgent({
  persona: "You are a support engineer who learns each customer's environment.",
  dreaming: {
    trigger: "step-count", // "off" | "step-count" | "compaction-event"
    behavior: "auto-launch", // "reminder" | "auto-launch"
    stepCount: 25,
  },
});

That compaction-event trigger is the important idea. The moment you are about to throw away detailed history is exactly the moment to decide what deserves to be kept.

What Letta gets right: a clear split between "always visible" and "available on request", and a deliberate step that runs when context is compressed. Because memory is git-versioned, every change the agent makes to its own memory is inspectable and reversible.

What to watch: self-editing memory means the agent decides what matters. That fits a personal assistant that learns your environment. In a companion app, users often want to decide for themselves which details are sacred.

How we do it at Sipplay

Our users are not configuring an agent. They are in the middle of a story, and they notice instantly when a character forgets a promise. So we combined ideas from both camps, and added one control we think is essential for companion apps: the user decides what is pinned.

Our context for each reply has four layers:

1. Pinning: the user's version of system/

Pinned Memory is our equivalent of Letta's system/ files, with one difference: the user controls it. When someone pins "Mira holds the archive key now" or "never call me by my old name", that text goes into the context sent to the model on every turn, however long the conversation gets.

This gives a guarantee retrieval cannot give: the detail is always present. It does not guarantee the model always uses it correctly. That is still up to the model, and we say so in our product copy. But it removes an entire class of failure: "the fact existed, but nothing fetched it."

Pins also sidestep the compaction problem entirely. Because they are injected separately on every turn, summarizing the conversation cannot erase them.

2. Compaction: summary and cards are different jobs

When a conversation window closes, an extraction step writes two things from the same messages: a new medium-term summary, and a set of long-term memory cards.

They do different jobs. The summary keeps the flow of the story. A card has to make sense on its own when it is retrieved weeks later, with no surrounding context.

Our evaluations taught us two lessons here:

• Unfinished plans fall through the cracks. In our English extraction tests (ten windows, 200 messages, 28 reference cards), models sometimes kept an upcoming trip in the summary but did not write it as a card. The next summary rewrite then dropped it, and the plan was gone. Our rule now is that unfinished plans which matter beyond the current window become cards, and get completed or expired later.
• Recall is not the only metric. An extractor that writes every possible card scores well on recall and badly on everything else, because each extra card is a distractor at retrieval time. We score precision as well: how much of what was written deserved to be kept.

One practical finding: two strong extraction candidates dropped to 80% and 60% valid JSON once we applied a 2,048-token output cap, because their reasoning used up the budget. Since extraction runs in the background after a window closes, a slower but reliable model is acceptable there. The same latency before every reply would not be.

This is the same instinct as Letta's compaction-triggered dreaming: the compression step is where you decide what survives, so do it deliberately.

3. Retrieval: decide before you search

Before any vector search, a small model decides whether this turn needs long-term memory at all, and if so, writes a retrieval plan:

{
  "need_retrieval": true,
  "memory_types": ["relationship_event"],
  "entities": ["user", "Lin"],
  "time_scope": "most_recent",
  "search_query": "reason and outcome of the user's most recent argument with Lin",
  "confidence": 0.93
}

The "no" cases matter as much as the "yes" cases. If the answer is already in the recent messages or the summary, searching again only adds a chance of pulling in an outdated record. In our 60-probe English test, the best candidate triggered retrieval on 29 of 30 cases that needed it and correctly skipped all 30 that did not. Others searched when the summary already had the answer.

Then, unlike Mem0's ADD-only approach, our cards carry a status. Retrieval filters by user, character, namespace, and active, stale, or deleted state before ranking. When a meeting moves from Thursday to Friday, the Thursday card is marked stale rather than left to compete.

We learned why the hard way. In an embedding test, two of three models ranked the outdated Thursday booking above the Friday update. Both were "relevant". Similarity alone cannot tell you which fact is still true.

Side by side

None of these is "the right answer". They make different bets about who should control memory: the developer, the agent, or the user.

What a personal agent should borrow

If you are building a personal agent, whether a companion, an assistant, or a coding helper that knows your setup, here is the checklist I would start from:

1. Have a small, explicit always-in-context set. Hard constraints, identity, and current commitments. Keep it short, because everything in it competes for attention on every turn. Let the user see and edit it.
2. Treat compaction as a decision point, not cleanup. Run extraction when you compress, not only on a timer. Check specifically for plans and promises that are about to fall out of the summary.
3. Store time and status, not just text. Whether you go ADD-only with temporal ranking like Mem0, or mark records stale like we do, the system must be able to tell "was true" from "is true".
4. Decide whether to retrieve before you retrieve. Test the negative cases: turns where the answer is already in context.
5. Evaluate each layer separately. Extraction quality, the retrieval decision, ranking, and final answer quality fail in different ways. One end-to-end score hides which one broke.
6. Isolate users and characters with zero tolerance. A memory from the wrong person is worse than no memory.

Where we are now

These ideas are running in Sipplay today. Pinned Memory is something users can try directly: pin a detail in a long story and watch whether the character keeps track of it fifty messages later. The evaluation figures in this post come from our internal development sets, not an independent benchmark, and we keep testing as we change models.

If you are working on memory for agents, I would love to compare notes. What do you keep always in context, and what happens to it when the conversation gets compressed?

Sources: Mem0 README and Letta README and memory docs, checked November 2, 2026. Star counts change daily. Sipplay figures are from internal development evaluations; the embedding corpus was Chinese, and the intent and extraction tests used English scenarios.