Long Term Memory in LangGraph

Summary (Long Term Memory in LangGraph)

This video is a step-by-step guide to adding long-term memory to LangGraph agents so chatbots can personalize responses across sessions. It builds from fundamentals (memory types) into a production-grade implementation.

---

1) What “Long Term Memory” means for an LLM chatbot

Users interact with an LLM chatbot in separate conversation threads, so the model otherwise lacks continuity about the user.

The solution is a persistent memory store (e.g., database-like storage):

• While the user chats, the system extracts user-specific facts from messages.
• Those facts are written into the memory store.
• On future chats, the agent first retrieves relevant memories and injects them into the LLM’s prompt so responses are tailored to the user (e.g., name, profession, preferences).

Key property emphasized:

Long-term memory is persistent: it survives closing the chat window and (with proper storage) even restarting the program.

---

2) LangGraph architecture for long-term memory: memory stores

The video explains that memory storage is implemented via a Base Store abstraction that supports operations like:

• Create/store memories
• Fetch/get memories
• Search memories
• Edit memories
• Delete memories

Then it compares concrete store implementations:

• In-memory store (RAM)
  • Easy for prototyping but volatile (lost on restart).
• Postgres store (persistent)
  • Used for production-grade systems; memories persist across restarts.
• Redis store (also mentioned)
  • Another persistence option.

---

3) Part 1: Coding tutorial — implementing memory store operations

The first major coding segment teaches:

• How to import and instantiate a memory store
• How to organize memories using namespaces (folder-like organization), e.g.:
  • ("users", "U1")
  • nested namespaces like:
    • ("users","U1","Profile")
    • ("users","U1","Preferences")
• How to use store methods:
  • put(namespace, key, value) to create memories
  • get(namespace, key) to retrieve a specific memory
  • search(namespace) to retrieve all memories in a namespace

Retrieval nuance: why semantic search is needed

• get works when the exact memory key is known.
• search can return everything in a namespace, but it becomes noisy when there are many memories.
• The video motivates semantic search:
  • Retrieve only memories relevant to the current query/context.

---

4) Semantic search (embedding-based long-term memory retrieval)

The video shows semantic retrieval using an embedding model:

• Uses OpenAI text embeddings (e.g., “text-embedding-3-small”).
• The embedding model is passed into the memory store so it can store vector embeddings.
• During search:
  • The query is embedded
  • The store returns the closest matching memories
  • A limit controls how many results come back

Outcome: Instead of injecting 100 unrelated memories, the agent injects only the top relevant ones (e.g., travel plans).

---

5) Part 2: Build a LangGraph chatbot that uses existing long-term memory

A simple LangGraph workflow is built:

• A chat node that:
  1. Reads user-specific memories from the store (based on user ID → namespace)
  2. Formats them into the system prompt
  3. Calls the LLM to respond

System prompt template details

The “system prompt template” is designed to:

• Use the user name when available
• Reference project/tool context
• Match preferred tone and avoid generic responses
• Suggest follow-up questions based on the user profile

Demonstration: The bot uses stored facts like the user’s name, profession, and preferences to personalize responses.

---

6) Extend the chatbot to create new memories while chatting

Next the video adds a workflow that can write new memories:

• A Remember node (before chat) that:
  • Uses an “extractor LLM” to decide whether the latest user message contains stable, user-specific information worth saving
  • Uses structured output with a “Memory Decision” (boolean + extracted memories)
  • If worth saving, stores extracted memories in the memory store

Deduplication problem (major flaw addressed)

Issue shown: Without deduplication, repeated user statements create duplicate memory entries.

Fix strategy taught:

• When extracting from the latest message, also pass in existing memories (or compare against them).
• The extractor returns memory items with metadata indicating whether each extracted memory is:
  • new vs already existing
• Only memories marked as new are stored.

---

7) Final merged workflow: real-time memory + personalized chat

A combined LangGraph design is presented:

• Remember node
  • Extracts and stores new memories (with deduplication)
• Chat node
  • Retrieves relevant memories and injects them into the prompt
  • Generates a personalized response

Demonstration: After user messages like name, teaching context, and favorite language, memories are created and reused. Messages that don’t add new info don’t create new memories.

---

8) Persistence upgrade: RAM vs Postgres (production-grade)

A critical limitation is highlighted:

• The basic memory store used earlier is RAM-based, so restarting clears memory.

Solution: Replace the in-memory store with a Postgres-backed store.

Practical setup guidance includes:

• Install Docker Desktop
• Run the Postgres container (via Docker commands)
• Use the Postgres connection URL in the LangGraph code

Validation:

• Restart the program
• Verify stored memories still exist

Conclusion:

Postgres storage provides persistent long-term memory suitable for production.

---

Main speakers / sources

• Nitesh (channel host; repeatedly introduced as “my name is Nitesh”)
• LangGraph (framework used in the tutorial)
• OpenAI embeddings (used for semantic search)