Summary of "Every Claude Code Memory System Compared (So You Don't Have To)"

Overview

This video compares “Claude Code memory systems”—how Claude retrieves the right context at the right time—and builds them into six escalating levels. Each level is grouped by:

Where memory lives (storage format/locations)
How Claude accesses it (retrieval/injection mechanism)

Core framing (the shared problem)

All memory systems answer the same question:

When you give Claude Code a task, how does it pull the right context at the right time?

Differences between levels come down to:

Storage location/format (Markdown files, vector/semantic chunks, local SQL/vector DB, cloud DB, etc.)
Retrieval mechanism (auto-loaded context vs. database search vs. hook-based injection vs. cross-tool querying)

Level 1 — Native Claude Code memory (Markdown “index + rules”)

Built-in: claw.md + memory.mmd (automemory)

`claw.md` (project folder rules)

A plain Markdown file stored in the project.
Used like an always-loaded “system prompt”-style context at session start.
Can be placed at different directory depths (project/root/subproject) and inherits downward, with local overrides taking precedence.
Common mistake: stuffing too much into claw.md.
- Claude “burns” the context window reading it every session.
- Guidance: keep claw.md under ~200 lines.
- Move large artifacts (e.g., brand voice) into separate external files and reference them from claw.md.

Automemory (`/memory`, `memory.mmd`)

Runs on the machine and creates a project-level memory structure.
Uses an index (memory.mmd) pointing to smaller memory files rather than dumping everything into one giant file.
Demonstrated with:
- project and user memory concepts (hierarchical “claw.md equivalents” at different levels)
- No memory folder in an empty project → automemory generates/updates structure only when needed.

Context window limitation (“context ro”)

The video ties better memory to context recall failure: as context grows, models can’t reliably “remember” everything they saw.

Mention of Anthropic work-in-progress (“Chyros”)

References a leaked/unreleased framework idea:

An always-on demon consolidating notes “while you sleep.”
Not public yet, but suggests native improvements.

Level 2 — More reliable structured memory injection (hooks + reorganize)

Goal: improve reliability and keep the built-in memory structure from becoming messy.

What Level 2 adds

Based on a concept involving John Connelly’s guide/prompt to create a structured memory system.
Uses Claude Code session start hooks to auto-inject a memory index (not the whole dataset) at the right time.

Memory structure created

Adds a global directory with:

memory.mmd (general/cross-project facts)
general.mmd (cross-project conventions)
Domain/topic files (one Markdown file per topic)
Tool-specific files (e.g., slack.md for tool config/workarounds)
Project memory files (per-project consolidation)

How it’s maintained (reorganize workflow)

Uses an AI-assisted routine:

initialize memory structure
plan changes
run reorganize memory to:
- remove duplicates/outdated entries
- merge related memories
- split files covering too many topics
- restore entries (including date-based organization)
- update the memory.mmd index

Hook-based injection via `settings.json`

Adds a settings.json hook (described as pre-tool use) that runs a script like pre-toolmemory.sh.
The index is injected automatically before tool calls / early in session, including for sub-agents.

Team sharing potential

Suggests synchronizing domain memory files between teammates via shared folders.

Scaling warnings

Level 2 is “good enough” initially, but the video claims it can degrade when memory grows:

keyword search becomes less reliable
giant index files become hard to read efficiently

Level 3 — Semantic memory with Memsarch (vector search + auto injection)

Goal: solve Level 2 scaling via better search and retrieval efficiency.

Based on OpenClaw memory architecture

Inspired by OpenClaw’s three-layer approach:

Long-term durable memory.mmd (facts/preferences/decisions)
Daily notes (one file per date; recent notes auto-loaded)
Dreaming (background process promotes recurring items to long-term; discards stale)

Memsarch (plugin)

A plugin/library that ports OpenClaw-like memory to Claude Code.

Designed for a Markdown-first approach:
- similar file structure: memory.mmd + daily notes
Uses semantic vectors:
- stored as chunks with embeddings
- retrieval is meaning-based, not keyword-based

Auto-inject top matches

Uses hooks (example: user prompt submit) to inject top 3 semantic matches into prompt context automatically.

Installation approach (as described)

Install memsearch plugin (2-line install approach)
Enable via settings.local.json
Reload plugins

Verification method

Confirm daily note creation (e.g., ls in the memory folder).
Use a built-in skill/command like “memory recall” to verify semantic retrieval.
Emphasizes similarity to an “agentic OS” pattern: daily logging + long-term memory.

Alternative mentioned: Claude Mem

Another plugin that:

captures/compresses Claude coding session activity into summaries/timeline/observations
includes dashboards/team collaboration/cost/privacy labels

Critique in the video:

more “MCP tool call”-driven (Claude must decide to search via tool)
less transparent than Markdown-readable memory
potentially overkill compared to Memsarch

Level 4 — Word-for-word conversation recall with Mem Palace

Goal: find the exact words previously used (not summarized).

Key properties

Described as a proper RAG system storing verbatim conversation text.
Claims high benchmark performance and very fast retrieval.

How Mem Palace stores memory

Uses a “memory palace” metaphor:

a symbolic index written in an “AA dense symbolic dialect” (as described in the video)
nested pointers: wing → room → drawer
pointers map to specific stored text locations

Storage design

Uses two databases:
- SQL for entities/relationships
- Chroma (vector DB) for searchable chunks
Local-only storage (on the machine).
Background hooking via Claude Code:
- filing + indexing happen on session end / lifecycle events (e.g., session end, pre-compaction, etc.)

Install/initialize/mining

One-command installation:
- creates me palace folder structure (wings/rooms/drawers)
- registers hooks in settings.json
Supports retrospective mining of old sessions.

Tradeoff called out

Retrieval is fast and exact, but stored data may not be human-readable “verbatim in Markdown” at rest.

Level 5 — Cross-source knowledge base: LLM Wiki (Carpathy-style)

Goal: connect and persist knowledge across many saved sources, not just chat/session recall.

When to use

For deep research / second-brain style knowledge linking over time:

articles, transcripts, podcasts, YouTube transcripts
client notes, etc.

The video suggests skipping this if you only consume content casually.

LLM Wiki approach (Carpathy)

Uses two folders inside a project:

raw/: source documents (read-only; Claude never writes here)
wiki/: Claude-owned generated knowledge in plain Markdown

Claude automatically:

maintains wiki structure
updates cross-references

Mentions it can work alongside an Obsidian knowledge graph workflow.

Alternatives discussed briefly

Recall (hosted service)
- no Obsidian setup
- hosted/rented data vs local ownership
- seems optimized for content consumption more than operational/agent memory
- pricing implications
LightRAG
- enterprise-grade knowledge graph + entity extraction/heavy retrieval
- framed as overkill for most business owners
- more for research/knowledge graphs than operational memory

Level 6 — Cross-tool shared memory: Open Brain (Postgres-backed)

Goal: make memory portable and available to multiple AI tools in real time (phone chat, desktop Claude, Cursor, etc.).

What Level 6 enables

One shared “brain” so different AI tools see the same memory automatically.
Portability claim: when new tools appear, connect them to the same DB without migrating memory.

Implementation model (Open Brain / Nate Jones concept)

Uses a Postgres database (described as hosted on Supabase).
Single table: thoughts
- each row contains:
  - text content
  - embedding vector
  - tags
  - timestamp
Uses semantic-search-capable DB extensions.
An MCP server acts as a front door so various clients/tools query it.

Tradeoffs

longer setup + more complex architecture
external DB latency (not fully local)
ongoing monthly cost (video suggests ~10–30 cents/month on free tier; notes higher than basic local levels)

Alternatives mentioned

Mem0 (mem.ai) / “Mem0ero”
- production-oriented, widely used
- quicker setup (<1 minute)
- cross-tool memory layer
- downside: data lives on their servers (ownership/risk tradeoff)

Overall stance in the video:

Open Brain: more control/portability/ownership, but more complexity and latency
Mem0: easier to start, potentially better for shipping products, but hosted data and recurring pricing/constraints

“Which one should you pick?” (video guidance)

Starting out: Level 1 (claw.md + memory.mmd correctly)
Using Claude Code for a while / medium scale: Level 2 (John’s hook + reorganize)
Months of work / losing old decisions: Level 3 (Memsarch semantic retrieval) or Level 4 (Mem Palace verbatim)
Deep topic research / knowledge graph: Level 5 (LLM Wiki / Carpathy pattern)
Shared memory across multiple tools with portability/ownership: Level 6 (Open Brain)

Stacking

Levels 1–3 can stack together with similar folder structures and complementary search/injection behavior.
Personal implementation claim: implementing up to Level 3 in their “agentic operating system,” using OpenClaw conventions plus semantic search + hooks.

Main speakers / sources

Speaker (primary): YouTube narrator/author of “Every Claude Code Memory System Compared (So You Don’t Have To)” (name not provided in subtitles)
Referenced sources/creators:
- John Connelly (Level 2 concept/article/prompt)
- Andrej Karpathy (LLM Wiki / “Carpathy” resources)
- OpenClaw (memory architecture inspiration)
- Zillus team (Memsearch plugin)
- Me Palace official (Mem Palace framework)
- Nate Jones (Open Brain concept)
- Anthropic (Claude Code native memory efforts; “Chyros” mention)
- Mentions of Pavl Hurin (inspiration for Level 2 concept)

Share this summary

Is the summary off?

If you think the summary is inaccurate, you can reprocess it with the latest model.

Summarize another video