Agentic Memory Explained: How LLM Agents Remember Across Sessions

Ask a large language model what you told it last week and, unless you paste the transcript back in, it has no idea. The model itself is stateless. Every request starts from zero, and the only thing standing between an agent and total amnesia is whatever text you can cram into the context window. That works for a single chat. It falls apart the moment you want an agent that improves over days, remembers a user's preferences, or picks up a long-running task where it left off.

Agentic memory is the layer that fixes this. It is the set of mechanisms an AI agent uses to store what happened, decide what is worth keeping, and pull the right piece back at the right moment, all without a human re-pasting history. This guide explains what agentic memory actually is, why the context window is not a substitute, the kinds of memory agents use, and how leading systems make agents remember across sessions.

What Is Agentic Memory?

Agentic memory is a persistent, external store of an agent's experiences and knowledge, combined with the logic that reads from and writes to it during a task. The word "agentic" matters here. A plain database is passive: it holds rows until something queries them. Agentic memory is active. The agent itself decides what to record, when to update an old fact, and which memories are relevant to the current step. Memory becomes part of the reasoning loop rather than a bolt-on lookup.

It helps to separate two ideas that often get blurred. Retrieval-augmented generation (RAG) pulls documents from a fixed corpus to ground an answer. Agentic memory records the agent's own history, its conversations, decisions, tool results, and reflections, then feeds those back later. RAG answers "what does the knowledge base say?" Memory answers "what have I already learned or been told?" Most production agents use both.

Why the Context Window Is Not Memory

A common assumption is that a bigger context window removes the need for memory. It does not. Three problems remain no matter how large the window gets.

First, cost and latency scale with the number of tokens you send. Replaying an entire multi-week history on every request is expensive and slow, and most of those tokens are irrelevant to the current step. Second, models struggle with very long inputs. Research on long-context behavior has repeatedly shown a "lost in the middle" effect, where information buried in the center of a long prompt is used far less reliably than material near the start or end. Third, and most decisively, the window resets. When a session ends, whatever was in context is gone. A new session begins blank.

For a deeper walkthrough of how memory fits into the broader agent stack, our AI Agent Memory: A Developer's Guide covers the architecture end to end. This post zooms in on one question that guide raises: how does an agent actually carry knowledge from one session into the next?

The Types of Agent Memory

Most agent memory designs borrow a rough taxonomy from cognitive science. The categories are not rigid, but they are a useful way to reason about what to store and where.

Short-Term (Working) Memory

Short-term memory is the current context window: the active conversation, the last few tool outputs, and the scratchpad the agent is reasoning over right now. It is fast and immediate, but volatile. It disappears when the session ends or when older turns get pushed out to make room. Think of it as the agent's attention, not its knowledge.

Long-Term Memory

Long-term memory persists across sessions in an external store. It usually breaks into three subtypes:

Episodic memory records specific events: "the user asked for a refund on March 3rd," "the last deployment failed on the migration step." These are timestamped experiences the agent can recall later. Semantic memory holds distilled facts and preferences: "the user prefers Python," "this customer is on the enterprise plan." It is the general knowledge the agent has accumulated, stripped of the specific moment it was learned. Procedural memory captures how to do things: reusable skills, successful action sequences, and workflows the agent can replay instead of rediscovering. Our companion post on agent workflow memory digs into this last category, showing how agents extract reusable routines from their own past runs.

The practical point is that these types want different storage. Semantic facts fit a key-value or vector store. Episodic events fit a timeline or knowledge graph. Procedural skills fit a library of callable routines.

How Agents Remember Across Sessions

Making an agent remember across sessions comes down to a loop with four stages: encode, store, retrieve, and consolidate. Every serious memory system implements some version of this cycle.

Encoding: Deciding What to Write

Not everything is worth remembering. Storing every raw token creates a bloated, noisy store that is expensive to search and full of contradictions. So the first step is extraction. Many systems use the LLM itself to read a conversation and pull out the salient facts before writing them down. Mem0, for example, runs an explicit extraction stage that identifies key facts from a new exchange, then an update stage that decides whether to ADD a new memory, UPDATE an existing one, DELETE something now false, or do nothing (NOOP). That last part is important: good memory is as much about revising and forgetting as it is about adding.

Storage: Where Memories Live

Once extracted, a memory needs a home. The two dominant choices are vector stores and knowledge graphs. A vector store embeds each memory as a numeric vector so the agent can later find semantically similar items. A knowledge graph stores memories as entities and relationships, which is far better when the agent needs to reason about how facts connect or change over time. Zep takes the graph route with Graphiti, a temporally aware knowledge graph that tracks when each fact was valid, so an agent can answer "what was true then?" rather than only "what is true now."

Retrieval: Pulling the Right Memory Back

At the start of a new session or step, the agent queries its store for relevant memories and loads only those into context. Retrieval is usually driven by semantic similarity to the current query, but the best systems layer in additional signals. The influential Stanford "generative agents" work scored memories on a combination of relevance, recency, and importance, so a recent or significant event could outrank a merely similar one. This selective loading is what keeps the context window small while still giving the agent the knowledge it needs.

# A minimal agentic-memory loop
def handle_turn(agent, memory_store, user_msg):
    # 1. RETRIEVE: pull the most relevant past memories
    relevant = memory_store.search(query=user_msg, top_k=5)

    # 2. REASON: give the model only what it needs, not the full history
    context = build_prompt(memories=relevant, message=user_msg)
    response = agent.generate(context)

    # 3. ENCODE + STORE: let the model decide what is worth keeping
    facts = agent.extract_facts(user_msg, response)
    for fact in facts:
        memory_store.upsert(fact)  # ADD, UPDATE, or NOOP

    return response

Consolidation: Making Memory Smarter Over Time

The final stage is what separates a memory store from true agentic memory. Instead of leaving records frozen, the system periodically reorganizes them: merging duplicates, resolving conflicts, and summarizing clusters of related events into higher-level insights. The Stanford agents ran a "reflection" step that synthesized recent observations into abstract conclusions. LangMem uses a background manager to consolidate knowledge continuously so the agent's behavior stays consistent without blocking the main loop.

Agentic Memory in Practice: The Frameworks

You rarely build all of this from scratch. A cluster of purpose-built memory frameworks emerged over 2025 and into 2026, each making a different architectural bet.

• Mem0: an extract-then-update pipeline over a vector store (ADD/UPDATE/DELETE/NOOP). Best fit: personalization and low-latency retrieval.
• Zep: a temporal knowledge graph (Graphiti) that tracks fact validity over time. Best fit: temporal reasoning over evolving facts.
• LangMem: a background consolidation manager with tight LangGraph integration. Best fit: LangChain-based agents.
• Letta (MemGPT): OS-style paging between the context window and external memory. Best fit: agents that self-manage their own memory.
• A-MEM: self-organizing, interlinked notes that evolve as new memories arrive. Best fit: long, multi-session knowledge building.

Reported benchmarks favor different tools for different jobs. Mem0's authors report the lowest search latency of the group, with median retrieval around 0.15 seconds, which matters for interactive chat. Zep's graph shines when facts change and the agent must respect chronology. The right pick depends on whether your bottleneck is speed, temporal accuracy, or ecosystem fit.

A-MEM: Memory That Organizes Itself

One of the more interesting recent designs is A-MEM (Agentic Memory for LLM Agents), presented at NeurIPS 2025. It is worth a closer look because it pushes the "agentic" part of agentic memory further than most.

A-MEM borrows from the Zettelkasten method, a note-taking system built on densely linked index cards. When the agent forms a new memory, it does not just store raw text. It generates a structured note containing a contextual description, keywords, and tags, then links that note to related existing memories. Crucially, adding a new memory can trigger evolution: the system asks the model whether the nearest existing notes should have their own context, keywords, or tags updated in light of the new information. The knowledge network refines itself over time rather than growing into an undifferentiated pile.

The reported results are strong. On LoCoMo, a long-term conversation benchmark with dialogues averaging around 9,000 tokens across up to 35 sessions, A-MEM reports up to a six-fold improvement on complex multi-hop reasoning while cutting memory-operation token usage by 85 to 93 percent versus baselines. The token savings come directly from retrieving small, well-organized notes instead of replaying long histories.

Design Patterns and Common Pitfalls

A few lessons show up again and again once memory is in production.

Store distilled facts, not raw transcripts. Extraction is not optional at scale. A store full of verbatim chat logs is slow to search and riddled with contradictions. Let the model summarize before it writes.

Give memory an eviction and update policy. Memory that only grows will eventually contradict itself and blow up retrieval cost. The ability to UPDATE and DELETE is what keeps a store trustworthy. Recent benchmarks specifically probe how memory systems behave under conflicting facts, and the ones without a clear update policy degrade fastest.

Scope memory correctly. A user's private preferences should not leak into another user's session. Namespace memory by user, by agent, or by tenant, and be deliberate about what is shared versus isolated.

Finally, treat retrieval quality as the thing that makes or breaks the system. An agent with a perfect store and poor retrieval behaves exactly like an agent with no memory at all. Tune your top-k, blend similarity with recency and importance, and evaluate retrieval as its own component.

Where to Go Next

Agentic memory turns a stateless model into something that behaves like it learns. The pattern is consistent across every serious system: encode selectively, store in a structure that fits the data, retrieve only what is relevant, and consolidate in the background so the store gets smarter instead of just bigger. Start simple with a vector store and an extraction step, then add graph structure or self-organizing notes as your agent's memory needs grow.

If you are building in this space, start with our AI agent memory developer's guide for the full architecture, then read how agent workflow memory lets agents reuse their own past steps. More engineering deep dives live on the Levelop blog.

References

Xu et al., "A-MEM: Agentic Memory for LLM Agents," arXiv:2502.12110 (NeurIPS 2025). arxiv.org/abs/2502.12110

Chhikara et al., "Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory," arXiv:2504.19413. arxiv.org/abs/2504.19413

Park et al., "Generative Agents: Interactive Simulacra of Human Behavior," arXiv:2304.03442. arxiv.org/abs/2304.03442

Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," arXiv:2307.03172. arxiv.org/abs/2307.03172

Zep / Graphiti temporal knowledge graph documentation. getzep.com