An AI agent without memory is a chatbot that forgets you between turns. An agent with memory can carry context across conversations, learn from past interactions, and build user-specific behavior. In 2026 there are three distinct memory patterns in use across production agent systems, and picking the wrong one costs either reliability or latency. This article walks through the three, when to use each, and the libraries that implement them.
What agent memory actually is
Memory in an agent context is not one thing. It is three different capabilities that look similar from outside but have very different implementation shapes.
- Conversation memory (short-term): the message history within a single session. Starts fresh when the user opens a new chat.
- Semantic memory (long-term): facts the agent should remember across sessions. “The user prefers metric units.” “The user’s dog’s name is Rex.”
- Episodic memory (long-term): specific past interactions the agent can recall when relevant. “Last Tuesday the user asked about returning a product.”
Most production agents need all three. Treating them as one monolithic memory store is the common failure mode that leads to agents that remember too much, too little, or the wrong things.
Pattern 1: Conversation memory (short-term)
The simplest and most essential. Store the message history in your pipeline state. Pass it with every LLM call.
from langgraph.checkpoint.postgres import PostgresSaver
from langgraph.graph import StateGraph
checkpointer = PostgresSaver.from_conn_string("postgresql://user:pass@host/db")
graph = StateGraph(AgentState)
# ... define nodes
app = graph.compile(checkpointer=checkpointer)
# Each conversation gets a thread_id
config = {"configurable": {"thread_id": "user-123-session-abc"}}
app.invoke({"messages": [HumanMessage(content="Hi")]}, config=config)LangGraph’s checkpointer persists the full state (including message history) between runs. The same thread_id picks up the conversation where it left off. Switch to a different thread_id for a new conversation.
Watch out for context explosion. A long conversation balloons the message history until you hit the model’s context limit. Mitigate with summarization: after every N turns, replace the oldest messages with a system message that summarizes them. LangChain ships ConversationSummaryMemory for this.
Pattern 2: Semantic memory (long-term facts)
Store user-specific facts in a key-value store (Redis, Postgres, DynamoDB). Retrieve them on every conversation start and inject into the system prompt.
def get_user_facts(user_id):
return db.get_json(f"user:{user_id}:facts")
# {"units": "metric", "pet_name": "Rex", "tone_preference": "casual"}
def build_system_prompt(user_id):
facts = get_user_facts(user_id)
fact_lines = [f"- The user prefers {k}: {v}" for k, v in facts.items()]
return f"You are a helpful assistant.\n\nAbout the user:\n" + "\n".join(fact_lines)Writing facts to memory is the harder part. Three approaches:
- Explicit tool call: give the agent a
remember_fact(key, value)tool. The agent decides when to remember based on the conversation. - Background summarizer: after each conversation, run a separate LLM call that extracts new facts from the transcript and writes them to the fact store. Decoupled from the agent, runs async.
- Hybrid: both. Agent can remember explicitly during the call; the background job catches what the agent missed.
Pattern 3: Episodic memory (searchable past)
Store every past conversation as an embedding in a vector database. When a new conversation starts, retrieve the top-k most relevant past exchanges and inject them into the prompt as context.
from langchain_openai import OpenAIEmbeddings
from langchain_pinecone import PineconeVectorStore
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
memory_store = PineconeVectorStore(
index_name="agent-episodic-memory",
embedding=embeddings,
namespace=user_id, # per-user namespace isolates memories
)
def recall_relevant(user_id, current_query, k=3):
results = memory_store.similarity_search(current_query, k=k)
return "\n\n".join(f"Past interaction: {r.page_content}" for r in results)
def end_conversation(user_id, transcript):
memory_store.add_texts([transcript], metadatas=[{"timestamp": now()}])Episodic memory is the most expensive pattern (one embedding call per past exchange plus one retrieval per query) but gives the agent a “I remember when you asked about X” capability that users notice.
Combining the three in production
A real production agent uses all three:
- Conversation memory for within-session continuity (LangGraph checkpointer)
- Semantic memory for user profile and preferences (Postgres JSONB column)
- Episodic memory for recalling specific past exchanges when relevant (Pinecone with per-user namespace)
Compose them in the system prompt:
def build_full_prompt(user_id, current_query):
facts = get_user_facts(user_id)
past = recall_relevant(user_id, current_query, k=3)
return f"""You are a helpful assistant.
About the user:
{format_facts(facts)}
Relevant past interactions:
{past}
Respond to the user's current message.
"""Order matters: facts first (stable, short, high-signal), past interactions second (longer, retrieved per query). The conversation memory is handled separately by LangGraph’s checkpointer; you do not inject it into the system prompt because it is in the messages array.
Memory frameworks to consider
- LangGraph checkpointer: handles conversation memory (short-term) with Postgres, Redis, or SQLite backends. First-class.
- Mem0: open-source memory layer built specifically for AI agents in 2026. Handles semantic + episodic in one API. Works with any LLM provider.
- Zep: hosted memory service with a Python SDK. Fact extraction and vector retrieval built in. Paid tiers for production.
- LlamaIndex memory module: short + long-term memory with multiple backend options. Good fit if you are already on LlamaIndex.
- Build-your-own: Postgres + Pinecone + your own fact extractor. Most flexible, most maintenance.
Common failure modes
- Memory bloat: storing everything the user ever said. Fact store becomes a dumping ground; relevant facts drown in noise. Fix: cap at 20 to 50 facts per user and prune oldest unused.
- Stale facts: user says “I live in Austin now” six months after the fact store captured “I live in Boston.” Fix: timestamp every fact and prefer newer ones on conflict.
- Cross-user contamination: forgetting to namespace memory stores leads to User A seeing User B’s memories. Fix: namespace by user_id everywhere.
- Privacy leaks: episodic memory stores full conversation transcripts. Users may want to delete these on request. Fix: build the delete-my-memory endpoint before you launch.
Frequently Asked Questions
Official documentation
Do I need all three memory patterns?
For a simple assistant, conversation memory alone is enough. For a personalized assistant, add semantic memory. For an assistant that references past conversations (“as we discussed last week”), add episodic memory. Start simple and layer on patterns as the use case demands them.
How is agent memory different from RAG?
RAG retrieves information from a shared knowledge base (docs, product catalog) that is the same for every user. Agent memory stores user-specific state that varies per user. Both use vector search in similar shapes, but the data lifecycle (shared vs per-user) and update patterns (ingestion pipeline vs end-of-conversation write) are different.
What is Mem0 and should I use it?
Mem0 is an open-source agent memory layer that landed in 2024 and matured through 2026. It handles fact extraction, deduplication, and semantic retrieval behind one API. Worth adopting if you want managed memory without building the three-pattern stack yourself. For a startup, Mem0 saves 2 to 4 weeks of memory infrastructure work.
How much does agent memory cost at scale?
Semantic memory is cheap (small Postgres queries). Episodic memory costs scale with vector-store operations and embedding API calls, typically $0.01 to $0.05 per user per month at moderate usage. Conversation memory stored in Postgres is nearly free. Full stack memory for 10,000 active users is under $100 per month in infrastructure.
Can memory contain private data for compliance (HIPAA, GDPR)?
Yes but with care. Semantic and episodic memory often contain personal data. Build consent, deletion, and audit flows from day one. Pick a vector database that supports data-residency guarantees if you serve EU users. For HIPAA, self-host the memory stack behind your HIPAA-compliant infrastructure.
How do I debug when memory misbehaves?
Trace memory operations with LangSmith or OpenTelemetry. Log every fact write, episodic store, and retrieval. When an agent remembers the wrong thing, the trace shows you which retrieval returned which memory and why. Memory bugs are hard to catch without this visibility.
Leah Whynett Dela Pena
Technology Writer at PIES IT Solution
Leah Whynett Dela Pena is a technology writer at PIES IT Solution, author of 93 career and IT education guides at itsourcecode.com. Specializes in tech career paths (software engineering, web development, cybersecurity, IT analyst roles), degree and certification guidance, and IT industry how-to content for students and career changers.
Expertise: Tech Careers · IT Education · Web Development Careers · Software Engineering Careers · Cybersecurity Careers · IT Certifications · Career Guidance · View all posts by Leah Whynett Dela Pena →