Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 2 min read

Why Your AI Agent Keeps Forgetting (and Getting Slower): 4 Memory Options and 3 Context Fixes

Most AI agents have two memory problems. They forget what a user told them last week, and every turn costs more than the one before. Both come from the same place: how much of the conversation you send to the model, and

Most AI agents have two memory problems. They forget what a user told them last week, and every turn costs more than the one before.

Both come from the same place: how much of the conversation you send to the model, and what you keep between sessions.

In this episode of AI Agent Clinic, Google Cloud engineer Laxmi Harikumar and developer Miles Libbey fix both in Scribant, a voice to action agent Miles built in TypeScript with Agent Development Kit (ADK).

Which memory option should you use?

The episode compares 4 ways to give an agent memory, using one test. A user says "I'm in New York", later says "I moved to San Francisco", then asks "Where do I live now?"

Option What the model gets Tradeoff
Flat prompting The full chat history Easy to build, but tokens grow every turn
Vector RAG Every chunk that matches, old and new Saves tokens, but retrieval alone doesn't say which fact is current
Knowledge graph The current fact Tracks changes, but you build the extraction, schemas and update logic
Memory Bank in Gemini platform The current fact Extracts, deduplicates and updates facts for you in the background

RAG is still the right tool for searching a knowledge base. Memory is a different job: keeping facts about a user current as they change.

How do you keep context small in a streaming agent?

Scribant gets new transcript text every 2 seconds, so resending everything adds up fast. The episode adds 3 fixes, each for a different time span:

  1. Rolling window (this turn): the intent classifier reads only the last 40 words.
  2. Rolling summary (this session): a running summary stands in for everything said so far.
  3. Memory Bank (across sessions): only the extracted facts are stored and recalled later.

What did the demo measure?

On one short test conversation in the episode (about 200 spoken words):

  • The 40 word window used about 86% fewer classifier tokens than sending the whole speech stream.
  • The rolling summary came to 92 tokens.
  • Memory Bank stored about 24 tokens of facts, where ingesting the whole conversation would have taken about 1,400.

These are results from one demo, not benchmarks. Treat 40 words as a starting point and tune it to how your users talk. The token tracking code is in the repo (token-telemetry.ts) if you want to measure your own agent.

Jump to a section

  • 0:00 Intro
  • 1:51 Live demo: spoken thoughts become calendar events and tasks
  • 2:44 Architecture: the 3 stage ADK pipeline and the execution gate
  • 4:21 Memory options: flat prompting, RAG and knowledge graphs
  • 5:56 Wiring up Memory Bank in ADK
  • 9:01 The 40 word rolling window
  • 12:07 Dashboard results
  • 14:41 Takeaways

Code and docs

How do you handle memory in your own agents: full history, RAG, or something else? Tell us in the comments.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.