Agent memory and persistence

TL;DR. The model forgets everything between calls, so durable memory is something your code keeps outside the window and re-injects a relevant slice of each turn. A memory system is four operations: extract atomic facts from a turn, embed and store them, retrieve the top-k relevant facts for a new query, and invalidate the stale fact when a new one contradicts it. The discipline that makes it work is retrieve, do not dump: pull the few facts this question needs, not the whole history, and update stale facts in place rather than letting the store accumulate contradictions.

Contents

Chapter 1 made one fact unavoidable: the model is a pure function of its context, and between calls it remembers nothing. The "memory" a chat assistant seems to have is your application re-sending the conversation each turn. That works until the conversation outgrows the window, or until the user comes back next week and expects you to still know their name. At that point re-sending the whole history is either impossible (it does not fit) or wasteful (you pay for thousands of tokens to answer a question that needed three of them).

This chapter is about the durable answer: store the facts outside the window, and on each turn re-inject only the slice that this turn actually needs. That is the memory family from Chapter 1's table, and it relieves the capacity pressure by keeping the context lean no matter how long the relationship runs.

The purpose is to decouple what the assistant knows from what currently fits. The window is a fixed-size buffer; the relationship is open-ended. Memory bridges the two by keeping the durable record in your own storage and copying only the relevant part into the window per turn. The benefit shows up in two currencies. The first is correctness: a question like "which seat do I prefer?" is simply unanswerable on a fresh call unless the preference is in the window, so memory turns an impossible question into an answerable one. The second is cost: you stop paying to re-send thousands of tokens of history that this particular turn does not touch, and that saving compounds in an agent loop that re-sends its context every turn (Chapter 2).

Three use-cases drive most of the value, and they recur through the chapter:

  • Cross-session personalization. The assistant remembers a user's preferences, tone, and past decisions from one visit to the next, so the user does not restate their name, their diet, or their seat preference every session.
  • Long-running assistants. An agent working a task for hours keeps durable notes outside the window, so it does not lose the thread when the oldest turns get evicted to make room. This pairs with two later techniques: Chapter 11 summarizes the running transcript so the window does not overflow in the first place, and Chapter 12 stores learned procedures (how to do a recurring task) rather than the facts this chapter stores.
  • Customer histories. A support agent recalls a specific account's prior tickets and context without re-reading the whole record on every message.

One lesson learned governs all three, and the rest of the chapter is an argument for it: retrieve, do not dump, and invalidate, do not accumulate. Pulling the few facts a question needs is what keeps the window small; updating a stale fact in place (rather than appending the new value beside the old one) is what keeps the store from returning two contradictory answers at random. A memory that only ever grows is worse than no memory, because it makes the model confidently wrong.

Memory is not the window

The word "memory" gets used for two different things, and conflating them is the usual source of confusion.

Don't be confused. The context window is the model's working memory for one call: a fixed-size buffer, rebuilt every turn, that the model reads and then forgets. An external memory store is your durable record: a database, file, or vector index that lives in your code and survives across calls and sessions. The model never sees the store directly. Your code reads from the store, selects what is relevant, and writes it into the window. Retrieval (pulling a few facts out of the store) is the opposite of "send the whole history" (dumping everything back into the window). The whole skill is keeping the store complete and the window small.

Remember. The store can grow without bound; the window cannot. So the per-turn job is never "give the model everything it knows," it is "give the model the small slice this turn needs." A memory system is judged on retrieval (does the right fact come back?) and on invalidation (does a stale fact get replaced, not duplicated?), not on how much it can hold.

Two layers: always-on and retrieved

Before the four operations, one split is worth naming, because most real setups use both halves at once. Durable context comes in two layers that trade off differently:

  • The always-on layer is content you inject into every turn, unconditionally. In Claude Code this is CLAUDE.md: a hand-written file re-injected at the start of every session. It is the right home for the small set of facts that every turn should know (the build command, the house style, where the source lives). Its cost is fixed and paid every session whether or not the current task touches it, so it has to stay short.
  • The retrieved layer is the memory store this chapter builds: a much larger body of facts the agent learned over time, of which only the few relevant to this turn are pulled into the window. Its cost scales with the question, not with the size of the store, so it can grow large without bloating any single call.

The discipline is to put the baseline that is always relevant in the always-on layer, and the long tail of facts (the ones that matter to one question in fifty) in the retrieved layer. Put the long tail in CLAUDE.md and you pay for all of it every turn; leave the baseline to retrieval and you risk it not being pulled when you need it. We return to this split concretely in the worked Claude Code session below.

Why a small relevant slice, not the whole history

It is tempting to skip retrieval and just re-send everything the agent has ever seen. Two forces make that a bad trade, and both get worse over time.

The first is token cost. The full history grows every single turn, so an agent that re-sends it pays a bill that climbs without limit, and in a loop it pays that climbing bill on every following turn (Chapter 2). A retrieved slice, by contrast, is roughly constant: it is always "the handful of facts this question touches," whether the relationship is one day or one year old.

The second, and the one people underrate, is accuracy. A window stuffed with hundreds of mostly-irrelevant facts is harder for the model to reason over, not easier. The one fact that answers the question is buried among ninety-nine that do not, and the model has to find it. A tight, relevant slice is not just cheaper, it is also a clearer prompt: fewer distractors, fewer chances to anchor on the wrong detail. Retrieving the relevant slice is therefore the correct default for both reasons at once, which is why the four operations below all serve it.

So a memory system has to answer four operational questions, and the rest of this chapter is those four. Each one is a distinct step with its own failure mode:

  1. Extraction. When the user says something worth keeping, what exactly do you store? You turn a chatty turn into one or more atomic facts, each about a single thing. "My name is Dana and I live in Lisbon" is two facts (name is dana, lives in lisbon), not one run-on string, because a fact you can retrieve cleanly has to be about one subject. Store the whole sentence and a later query for the city drags the name along with it; split it and each fact retrieves on its own. A real system hands this job to the model: prompt it to read the turn and emit clean, self-contained statements.
  2. Embedding and storage. In what form, so you can find it again later? Each fact is turned into an embedding, a vector of numbers positioned so that texts about the same thing land near each other, and the vector is written to a store (a vector index in production) keyed by user or session. The embedding is what makes retrieval work by meaning rather than exact words: a query and a fact that share no literal words can still score as similar if they are about the same topic.
  3. Retrieval. Given a new question, which stored facts go into the window this turn? You embed the query, score it against every stored fact by similarity, and keep the top-k highest-scoring facts. Only those k enter the window. This is the step that keeps the window small and the prompt clear, and it is the reason the cost scales with the question rather than the conversation.
  4. Invalidation and update. When a new fact contradicts an old one, how do you avoid keeping both? You detect that the new fact is about the same subject as a stored one and replace the old value in place instead of appending a second, contradictory copy. Skip this and the store accumulates "favorite color is blue" and "favorite color is green," and retrieval returns whichever happens to rank higher, which is a coin flip. Invalidation is what makes memory trustworthy rather than merely large.

A memory store in 150 lines

The code below builds the smallest honest version of all four. It uses NumPy and the standard library only: no model call, no database, no network. Read it for the mechanics, not the scale; a production system swaps each piece for a stronger one, but the shape is the same.

Three terms it leans on, defined before you meet them:

  • An embedding is a list of numbers that stands in for a piece of text, arranged so that texts about the same thing land near each other in number-space. Real embeddings come from a trained network; ours is a cheap stand-in that hashes each word into one of 256 slots and counts it, so two facts that share words get overlapping vectors.
  • Cosine similarity measures how aligned two vectors are: 1.0 means they point the same direction (very similar), 0.0 means they are unrelated. We unit-normalize every vector (scale it to length 1), which makes the cosine just the dot product, $\cos(a,b)=a\cdot b$.
  • Top-k retrieval means: score every stored fact against the query, sort, and keep the best $k$. Only those $k$ enter the window, not the whole store.
"""A tiny agent-memory store: extract, embed, store, retrieve, invalidate.

The model is stateless between calls. Anything it should "remember" across
turns or sessions is state YOUR code keeps outside the context window and
re-injects on demand. This file builds the smallest honest version of that:

  EXTRACT   pull atomic facts from a user turn with simple rules.
  EMBED     turn a fact into a fixed-length vector with a hashing bag-of-words.
  STORE     keep (fact, vector) in a list.
  RETRIEVE  rank stored facts against a query by cosine similarity, top-k.
  INVALIDATE when a new fact contradicts an old one on the same subject and
            attribute (e.g. "favorite color"), replace the old fact in place.

numpy + stdlib only. No network, no model call. ~1.3 tokens/word for sizing.
"""

import re
import numpy as np

DIM = 256  # length of every embedding vector


# --------------------------------------------------------------------------
# EXTRACT: user turn -> list of normalized atomic facts
# --------------------------------------------------------------------------
# We look for a few first-person and "X is Y" shapes. A real system would use
# the model itself for this; the rules here are deliberately small so you can
# see exactly what becomes a fact and what does not. We split the turn into
# clauses first ("A and B" -> ["A", "B"]) so each fact stays atomic, and the
# value group stops at the first clause break instead of swallowing the rest.
_VALUE = r"([\w ]+?)(?: and |[.,!?]|$)"  # a value, ending at a clause boundary
_PATTERNS = [
    # "my favorite color is blue" -> "favorite color is blue"
    (re.compile(r"\bmy ([\w ]+?) is " + _VALUE, re.I), "{0} is {1}"),
    # "i prefer window seats" -> "prefers window seats"
    (re.compile(r"\bi prefer " + _VALUE, re.I), "prefers {0}"),
    # "i am vegetarian" / "i'm allergic to peanuts" -> "is ..."
    (re.compile(r"\bi am " + _VALUE, re.I), "is {0}"),
    (re.compile(r"\bi'm " + _VALUE, re.I), "is {0}"),
    # "i live in Lisbon" -> "lives in Lisbon"
    (re.compile(r"\bi live in " + _VALUE, re.I), "lives in {0}"),
    # "i work at Helios" -> "works at Helios"
    (re.compile(r"\bi work at " + _VALUE, re.I), "works at {0}"),
]


def extract_facts(turn):
    """Return a list of normalized fact strings found in one user turn."""
    facts = []
    for pattern, template in _PATTERNS:
        for match in pattern.finditer(turn):
            groups = [g.strip().lower() for g in match.groups()]
            fact = template.format(*groups)
            fact = re.sub(r"\s+", " ", fact).strip()
            if fact and fact not in facts:
                facts.append(fact)
    return facts


# --------------------------------------------------------------------------
# EMBED: fact string -> fixed-length unit vector (hashing bag-of-words)
# --------------------------------------------------------------------------
# An "embedding" is a list of numbers that stands in for a piece of text, built
# so that texts about the same thing land near each other. Real embeddings come
# from a trained network. Ours is a cheap stand-in: hash each word to one of DIM
# slots and count it. Shared words -> overlapping slots -> nearby vectors.
_WORD = re.compile(r"[a-z0-9]+")


# A handful of words carry no topic ("i", "is", "my", "the"): drop them so two
# facts are judged similar by their content words, not their grammar.
_STOP = {"i", "is", "am", "my", "the", "a", "an", "and", "to", "in", "of",
         "at", "for", "me", "you", "your", "what", "should"}


def _stem(word):
    """Crudely fold a trailing 's' so 'seat' and 'seats' hit the same slot."""
    if len(word) > 3 and word.endswith("s") and not word.endswith("ss"):
        return word[:-1]
    return word


def embed(text):
    """Hash the words of `text` into a DIM-length vector, then unit-normalize."""
    vec = np.zeros(DIM, dtype=np.float64)
    for word in _WORD.findall(text.lower()):
        if word in _STOP:
            continue
        # Python's hash is salted per process, so use a stable hash instead.
        slot = hash_stable(_stem(word)) % DIM
        vec[slot] += 1.0
    norm = np.linalg.norm(vec)
    if norm > 0:
        vec /= norm  # length 1, so the dot product below is a cosine directly
    return vec


def hash_stable(word):
    """A small deterministic string hash (FNV-1a), so runs are reproducible."""
    h = 2166136261
    for ch in word.encode("utf-8"):
        h = (h ^ ch) * 16777619 & 0xFFFFFFFF
    return h


def cosine(a, b):
    """Cosine similarity of two vectors: 1.0 identical direction, 0.0 unrelated.

    Both vectors are already unit length, so this is just their dot product.
    """
    return float(np.dot(a, b))


# --------------------------------------------------------------------------
# The store: facts, their vectors, and the subject/attribute key for updates
# --------------------------------------------------------------------------
class MemoryStore:
    def __init__(self):
        self.facts = []       # list of fact strings
        self.vectors = []     # list of embeddings, aligned with self.facts
        self.keys = []        # subject/attribute key, aligned, for invalidation

    def _key(self, fact):
        """The part of a fact that identifies WHAT it is about, minus the value.

        "favorite color is blue" and "favorite color is green" share the key
        "favorite color is", so the second should replace the first, not sit
        beside it. We key on the text up to and including the last " is ".
        """
        marker = " is "
        if marker in fact:
            return fact[: fact.rindex(marker) + len(marker)]
        # "prefers window seats" -> key "prefers", value "window seats"
        parts = fact.split(" ", 1)
        return parts[0] if len(parts) > 1 else fact

    def add(self, fact):
        """Store a fact, REPLACING any existing fact with the same key."""
        key = self._key(fact)
        for i, existing_key in enumerate(self.keys):
            if existing_key == key:
                old = self.facts[i]
                self.facts[i] = fact
                self.vectors[i] = embed(fact)
                return ("updated", old)
        self.facts.append(fact)
        self.vectors.append(embed(fact))
        self.keys.append(key)
        return ("added", None)

    def retrieve(self, query, k=3):
        """Return the top-k (fact, score) most similar to `query`."""
        if not self.facts:
            return []
        q = embed(query)
        scored = [(self.facts[i], cosine(q, self.vectors[i]))
                  for i in range(len(self.facts))]
        scored.sort(key=lambda pair: pair[1], reverse=True)
        return scored[:k]


# --------------------------------------------------------------------------
# Sizing helper: estimate tokens the way Chapter 2 does (words * 1.3)
# --------------------------------------------------------------------------
def est_tokens(text):
    return round(len(text.split()) * 1.3)


# --------------------------------------------------------------------------
# Demo
# --------------------------------------------------------------------------
def main():
    # ---- Session 1: the user tells us things across several turns ----
    session_1 = [
        "Hi! My name is Dana and I live in Lisbon.",
        "I prefer window seats and I am vegetarian.",
        "By the way my favorite color is blue.",
        "I work at Helios and my budget is 2000 dollars.",
    ]

    store = MemoryStore()
    print("=== Session 1: extract facts from each turn and store them ===")
    for turn in session_1:
        found = extract_facts(turn)
        for fact in found:
            status, old = store.add(fact)
            if status == "updated":
                print(f"  turn: {turn!r}")
                print(f"    UPDATED  {old!r} -> {fact!r}")
            else:
                print(f"  turn: {turn!r}")
                print(f"    stored   {fact!r}")
    print(f"\n  memory now holds {len(store.facts)} facts:")
    for fact in store.facts:
        print(f"    - {fact}")

    # ---- A new session. The model remembers NOTHING on its own. ----
    # The user asks something only answerable from stored memory.
    print("\n=== Session 2 (later): a question unanswerable without memory ===")
    question = "Which seat do I prefer?"
    print(f"  user asks: {question!r}")

    # BEFORE memory: the only context is this one question. No stored facts.
    print("\n  -- BEFORE: no memory injected --")
    print("     context contains only the question; the model has never")
    print("     seen 'Dana', 'window seats', or anything from session 1.")
    print("     Best it can do: ask the user to repeat their preference.")

    # AFTER memory: retrieve the few relevant facts and inject only those.
    print("\n  -- AFTER: retrieve top-k relevant memories and inject them --")
    hits = store.retrieve(question, k=3)
    for fact, score in hits:
        print(f"     score {score:.3f}  {fact}")
    injected = "\n".join(f"- {fact}" for fact, _ in hits)
    print("     injected memory block:")
    for line in injected.splitlines():
        print(f"       {line}")
    print("     With 'prefers window seats' in context, the model answers:")
    print("       'I'll book you a window seat.'")

    # ---- The saving: inject everything vs inject only the relevant slice ----
    print("\n=== The saving: whole history vs top-k memories ===")
    full_history = " ".join(session_1)
    full_tok = est_tokens(full_history)
    topk_tok = est_tokens(injected)
    print(f"  inject ALL session-1 history : {full_tok:4d} tokens")
    print(f"  inject top-{len(hits)} memories        : {topk_tok:4d} tokens")
    saved = full_tok - topk_tok
    pct = 100.0 * saved / full_tok
    print(f"  saved                        : {saved:4d} tokens ({pct:.0f}% smaller)")
    print("  (and history grows every turn; the top-k slice does not.)")

    # ---- Contradiction: retrieval returns the UPDATED fact, not the stale one ----
    print("\n=== Contradiction handling: the update wins ===")
    color_q = "what is my favorite color?"
    before = store.retrieve(color_q, k=1)[0]
    print(f"  current best answer for {color_q!r}: {before[0]!r}")
    print("  user now says: 'Actually my favorite color is green.'")
    for fact in extract_facts("Actually my favorite color is green."):
        status, old = store.add(fact)
        print(f"    {status.upper()}  {old!r} -> {fact!r}")
    after = store.retrieve(color_q, k=1)[0]
    print(f"  best answer now: {after[0]!r}")
    print(f"  memory still holds {len(store.facts)} facts (no duplicate color).")


if __name__ == "__main__":
    main()

Running it:

=== Session 1: extract facts from each turn and store them ===
  turn: 'Hi! My name is Dana and I live in Lisbon.'
    stored   'name is dana'
  turn: 'Hi! My name is Dana and I live in Lisbon.'
    stored   'lives in lisbon'
  turn: 'I prefer window seats and I am vegetarian.'
    stored   'prefers window seats'
  turn: 'I prefer window seats and I am vegetarian.'
    stored   'is vegetarian'
  turn: 'By the way my favorite color is blue.'
    stored   'favorite color is blue'
  turn: 'I work at Helios and my budget is 2000 dollars.'
    stored   'budget is 2000 dollars'
  turn: 'I work at Helios and my budget is 2000 dollars.'
    stored   'works at helios'

  memory now holds 7 facts:
    - name is dana
    - lives in lisbon
    - prefers window seats
    - is vegetarian
    - favorite color is blue
    - budget is 2000 dollars
    - works at helios

=== Session 2 (later): a question unanswerable without memory ===
  user asks: 'Which seat do I prefer?'

  -- BEFORE: no memory injected --
     context contains only the question; the model has never
     seen 'Dana', 'window seats', or anything from session 1.
     Best it can do: ask the user to repeat their preference.

  -- AFTER: retrieve top-k relevant memories and inject them --
     score 0.577  prefers window seats
     score 0.000  name is dana
     score 0.000  lives in lisbon
     injected memory block:
       - prefers window seats
       - name is dana
       - lives in lisbon
     With 'prefers window seats' in context, the model answers:
       'I'll book you a window seat.'

=== The saving: whole history vs top-k memories ===
  inject ALL session-1 history :   47 tokens
  inject top-3 memories        :   16 tokens
  saved                        :   31 tokens (66% smaller)
  (and history grows every turn; the top-k slice does not.)

=== Contradiction handling: the update wins ===
  current best answer for 'what is my favorite color?': 'favorite color is blue'
  user now says: 'Actually my favorite color is green.'
    UPDATED  'favorite color is blue' -> 'favorite color is green'
  best answer now: 'favorite color is green'
  memory still holds 7 facts (no duplicate color).

Read the four sections in order, because each one is a piece of the answer.

Extraction turns chatty turns into atomic facts. The rules are small and rule-based here: a few patterns for my X is Y, I prefer Y, I am Y, I live in Y, I work at Y. The turn "My name is Dana and I live in Lisbon" becomes two separate facts, name is dana and lives in lisbon, not one run-on string, because a fact you can retrieve has to be about one thing. A real system hands this job to the model itself, asking it to emit clean facts; the rules above are just enough to make the step concrete.

Retrieval is where the window stays small. In session 2 the user asks "Which seat do I prefer?" The query shares the word "seat" with the stored fact "prefers window seats" (after a crude singular/plural fold so seat and seats collide), so that fact scores 0.577 while the rest score 0.000. The system injects the top 3 and the model can answer. Note what did not happen: the budget, the employer, and the diet stayed in the store, out of the window, because this question did not touch them.

The saving is the number that justifies the whole apparatus. Dumping all of session 1 into the window costs 47 tokens here; injecting the 3 retrieved facts costs 16, about two-thirds smaller. On a four-turn toy that is a rounding error. The point is the trend: the full history grows every single turn, so by turn fifty it is hundreds of tokens of mostly irrelevant chatter, while the top-k slice stays roughly constant because it is always "the few facts this question needs." Memory turns a cost that scales with conversation length into one that scales with question complexity.

Invalidation is the part people forget, and forgetting it is how assistants end up believing two contradictory things. The user said blue, then later said green. A naive store appends both and retrieval starts returning whichever happens to rank higher, which is a coin flip. The fix is to give each fact a key: the part that says what the fact is about, minus the value. Both color facts share the key favorite color is, so the second one replaces the first in place. After the update the store still holds 7 facts, not 8, and a query for the favorite color returns green. The key is what lets you tell "a new fact about the same thing" (update) apart from "a fact about a different thing" (add).

What the real systems do

The toy above is the shape; three open projects are the production versions, and they make different bets about how memory should be structured. The bets are not cosmetic. Each one answers the four operations differently, and each is the right choice for a different question about what you need to remember.

  • Mem0 is the closest to what we built: a vector-first fact store you bolt onto any agent. It extracts atomic facts from the conversation with an LLM, embeds them, stores them in a vector database, and retrieves the relevant ones with a multi-signal ranking (not just one cosine score). It also does the invalidation step, deciding whether a new fact adds to, updates, or contradicts what is stored. Its model of the world is flat: a bag of current facts, scored by similarity to the query. That is exactly the right model when the question is "what does the assistant know about this user right now," which covers most personalization. It is the simplest of the three to reason about, and it is what the rest of this chapter runs. It reports strong numbers on the LongMemEval long-conversation benchmark to back the pitch that this beats stuffing the full history into the window. (One currency note: in April 2026 Mem0 changed its default pipeline to single-pass, add-only extraction; the update-or-contradict step described here is the design from its paper, and Chapter 41 covers what changed and why.)

  • Letta (the project formerly published as MemGPT) takes an operating-system view. Instead of one flat store, it gives the agent a memory hierarchy of tiers, modeled on how an OS arranges fast-but-small RAM above large-but-slow disk: a small core tier always in the window (the agent's persona and the most important facts), a recall tier for recent conversation, and a large archival tier in external storage. The twist is that the agent pages its own memory: it has tools to move information between tiers, reading an archival fact into the window when it needs it and writing one back out when it does not, exactly the way an OS pages memory between RAM and disk. So the model, not your retrieval code, decides what is in context at any moment. That is the right model when the agent's task is long and open-ended enough that no fixed retrieval rule fits, and you want the model to manage its own working set. The cost is that you are trusting the model to page well, which is more machinery than a flat store needs.

  • Zep, built on the Graphiti engine, stores memory as a temporal knowledge graph: facts are nodes and edges, and each edge carries a validity window, a pair of timestamps saying from when until when the fact was true. That extra dimension is the whole point and the thing the other two cannot do. A vector store answers "what is true now"; a temporal graph also answers "what was true in Q1," because a superseded fact is not deleted, it is closed off (its validity window is ended at the moment a newer fact replaced it) and kept. Where Mem0 overwrites on invalidation, Zep records the transition. When the history of a fact matters (an account's plan changed in March, a price was different last quarter), that history is queryable rather than lost. It is the subject of Chapter 10.

Read across the three, the axis is how much structure each puts on the four operations. Mem0 keeps them flat and current and is the easiest to operate. Letta layers them into tiers the model itself manages, trading machinery for autonomy on long tasks. Zep adds a time axis so invalidation preserves history instead of erasing it, trading simplicity for the ability to answer "what was true then." Pick by the question you need to answer, not by which is newest.

The three use-cases from the start of the chapter map onto these choices. Cross-session personalization and customer histories usually want a vector-first store like Mem0 (what does this user prefer now?); a long-running assistant on an open-ended task fits Letta's self-managed tiers; and any case where the history of a fact is itself the question (auditing, "what was the budget last quarter") wants Zep's validity windows. The lesson from the start of the chapter is the through-line: whichever you pick, memory has to invalidate stale facts, not just accumulate them, and retrieve a relevant slice, not dump the whole store.

With Claude

You can implement memory entirely on your side, the way the demo does: keep the store, run retrieval, and paste the selected facts into the prompt. The Anthropic SDK also offers a memory tool that lets the model drive instead. You declare it on the request,

# Illustrative: requires the anthropic SDK and an API key.
import anthropic
client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=[{"type": "memory_20250818", "name": "memory"}],
    messages=[{"role": "user", "content": "Remember that I prefer window seats."}],
)

and the model can then read and write files in a memory directory across calls. You supply the storage backend (the tool tells you which file to read, create, or edit; where those bytes live is your code), which keeps you in control of where durable user data is kept. This is the Letta idea in miniature: the model manages its own notes rather than your retrieval code deciding for it. Consult the claude-api skill before wiring it up; the tool name and backend interface are exact.

Using the real tool: commands and before/after proof

The 150-line store above shows the mechanics. Here is how you reach for the production version. Mem0 is the closest match to what we built, so it is the one to run first.

It is a Python package called mem0ai (the import name is mem0). Install it, create a Memory, add a couple of facts under a user_id, then search to get back only the relevant ones. This is follow-along: the library and an embedding key are not on this box, so the output is labeled as expected, not measured here.

pip install mem0ai

# Follow-along: requires mem0ai and an embedding/LLM key in the environment.
from mem0 import Memory

m = Memory()  # default config: an LLM extracts facts, embeds them, stores them in a vector DB

# Session 1: hand Mem0 raw conversation turns. It extracts the atomic facts itself.
m.add(
    [
        {"role": "user", "content": "Hi, I'm Dana and I live in Lisbon."},
        {"role": "user", "content": "I prefer window seats and I am vegetarian."},
    ],
    user_id="dana",
)

# Session 2 (a later call, even a later day): retrieve only what this question needs.
hits = m.search(query="Which seat do I prefer?", filters={"user_id": "dana"}, top_k=3)
for h in hits["results"]:
    print(round(h["score"], 3), h["memory"])

0.41 Prefers window seats
0.22 Lives in Lisbon
0.19 Name is Dana

That is the same four-operation shape as the demo, with each piece swapped for a stronger one. add() does the extraction: you pass whole turns and Mem0's LLM splits them into atomic facts (Prefers window seats, not the run-on sentence). It embeds and stores them in a vector database keyed by user_id, and it runs the invalidation step, deciding whether a new fact adds to or replaces an old one. search() does retrieval: it scores the stored facts against the query and returns the top few. The filters={"user_id": "dana"} argument is the scoping key, so one store can hold many users without their facts leaking into each other.

Before and after: the metric is injected-context tokens

The claim worth proving is that retrieval keeps the window small. The metric is the number of injected-context tokens: how many tokens of prior knowledge you paste into the prompt to answer one cross-session question. Count it the way you would in production, with Anthropic's token counter, which returns the exact input-token count for a set of messages.

# Follow-along: requires the anthropic SDK and an API key.
import anthropic
client = anthropic.Anthropic()

question = "Which seat do I prefer?"

# (a) Dump the WHOLE prior history into the prompt.
full_history = "\n".join(prior_turns)  # every message from every past session
before = client.messages.count_tokens(
    model="claude-opus-4-8",
    messages=[{"role": "user", "content": full_history + "\n\n" + question}],
).input_tokens

# (b) Inject only the top-k facts Mem0.search returned.
memories = "\n".join(h["memory"] for h in hits["results"])
after = client.messages.count_tokens(
    model="claude-opus-4-8",
    messages=[{"role": "user", "content": memories + "\n\n" + question}],
).input_tokens

print(before, "->", after)

3000 -> 60

Those two numbers are illustrative (expected for a relationship with a few dozen past turns, not measured on this box), but the gap is the whole point. Dumping the full history costs roughly 3000 tokens and climbs every session; injecting the three facts Mem0 retrieved costs about 60 and stays flat, because it is always "the few facts this question needs."

The correctness half of the proof is sharper than the token count. Without memory, the cross-session question is simply unanswerable: a fresh call has never seen "Dana" or "window seats," so the best the model can do is ask the user to repeat themselves. With memory, the one fact that matters (Prefers window seats) is in the window, and the model answers "I'll book you a window seat." That is the same before/after you can run on this box right now: the from-scratch store demo above prints exactly that flip (score 0.577 prefers window seats, then the booked-seat answer), which is the on-box proof that the Mem0 numbers stand in for.

The other two libraries install the same way. Letta (the MemGPT project) ships the server as pip install letta and the client SDK as pip install letta-client; you talk to a running agent that pages its own memory tiers. Zep is pip install zep-cloud, a hosted temporal graph you write conversations to and query for the facts that were true at a given time.

A worked Claude Code session

Claude Code is the same stateless model: between sessions it remembers nothing on its own. It gets durable memory from two layers, and they map onto the split this chapter started with. One is the always-on baseline, the other is per-fact retrieval.

The first layer is CLAUDE.md, a file you keep in the project. Claude Code reads it and re-injects the whole thing at the start of every session, so it is in the window from the first turn without anyone asking for it. That makes it the place for the small set of stable facts every turn should know: the build command, the house style, which directory the source lives in. It is hand-maintained, not learned, and it does not shrink: a 600-line CLAUDE.md costs those tokens on every session whether or not the current task touches them. So it is the opposite trade from retrieval. Use it for the baseline that is always relevant, not for the long tail of facts that matter to one question in fifty.

The second layer is for those long-tail facts: things the model learns about you over time, which should survive across sessions and be pulled in only when a turn needs them. That is a retrieval store, and Claude Code reaches it through an MCP server. MCP (Model Context Protocol) is a standard way to expose an external tool server that the agent can call; a memory MCP server exposes "save this fact" and "search my facts" as tools. Mem0, Zep, and Graphiti, the same three projects from the section above, each ship one. You register it once with claude mcp add, giving the command Claude Code should run to start the server:

claude mcp add memory -- npx -y @mem0/mcp

After that, "memory" is a set of tools the agent can call mid-conversation, the way it already calls a file-read or a shell command. Now the two-session walkthrough.

Session 1 (Monday). You are wiring up a database module and you tell the agent a durable preference:

you: For this project, always use asyncpg for Postgres, never psycopg2.
     We standardized on it last quarter.

  [agent calls the memory tool: save("prefers asyncpg over psycopg2 for
   Postgres in this project")]

claude: Got it. I've noted that you use asyncpg here. I'll reach for it
        whenever this project talks to Postgres.

The fact is now in the external store, keyed to you. The rest of the session (the file the agent read, the migration it wrote, the test output) stays in this session's window and goes nowhere durable.

Session 2 (Thursday, a fresh window). Days later, none of Monday's conversation is in context. You ask a question that depends on the preference but never restates it:

you: Add a function that fetches a user row by id from Postgres.

  [agent calls the memory tool: search("Postgres library preference")
   -> "prefers asyncpg over psycopg2 for Postgres in this project"]

claude: Here's the fetch using asyncpg, matching your project convention:

        async def fetch_user(pool, user_id):
            async with pool.acquire() as conn:
                return await conn.fetchrow(
                    "SELECT * FROM users WHERE id = $1", user_id)

The agent retrieved one fact, the asyncpg preference, and acted on it. It did not need Monday's whole transcript in the window: just the one stored line the question touched. That is the top-k retrieval from the from-scratch demo, running for real. Contrast it with no memory: a fresh Claude Code session with no CLAUDE.md entry and no memory server has never heard of asyncpg, so it writes the fetch with psycopg2 (a reasonable default) and you correct it again, the same correction you already made on Monday.

Under the hood the model drives this through Anthropic's native memory tool. You declare it on the request,

tools=[{"type": "memory_20250818", "name": "memory"}]

and the model issues read and write commands against a memory directory that you back with storage: the tool says which file to view, create, or edit, and where those bytes live is your code. An MCP memory server is one way to provide that backing store. Either way the model manages its own notes (the Letta idea from the section above, in miniature) instead of your retrieval code deciding for it.

The payoff is the same number the from-scratch store proved. Without memory, the cross-session question is unanswerable or wrong: the new session re-learns the preference every time. With memory, the one relevant fact enters the window and nothing else does, which is the 66% smaller injected context the demo measured. And because each stored fact carries a key, a later "actually, we moved to psycopg3" updates the asyncpg note in place instead of leaving the agent with two contradictory preferences to pick between at random. You can watch the size of what gets injected with /context, and the spend with /cost.

Further reading

The primary sources for the systems and ideas in this chapter, all real:

  • "MemGPT: Towards LLMs as Operating Systems" (Packer et al., 2023), arxiv.org/abs/2310.08560. The paper behind the OS-style tiered memory: core, recall, and archival tiers that the agent pages itself. Read it for the framing of context as a managed hierarchy rather than a flat store. The project lives on at github.com/letta-ai/letta.
  • Mem0, github.com/mem0ai/mem0. The vector-first fact store this chapter runs: extract with an LLM, embed, store, retrieve top-k, invalidate on contradiction. The repo's README and docs cover the add/search API and the LongMemEval results referenced above.
  • Zep, github.com/getzep/zep (built on the Graphiti engine at github.com/getzep/graphiti). The temporal-knowledge-graph store: facts as nodes and edges, each edge carrying a validity window, so superseded facts are closed off rather than deleted. The bridge to Chapter 10.
  • Anthropic's memory-tool documentation, in the Claude developer docs under tool use. The exact contract for the memory_20250818 tool: which commands the model issues against the memory directory, and what your backing storage has to implement. Consult the claude-api skill before wiring it up.
  • LongMemEval (Wu et al., 2024), arxiv.org/abs/2410.10813. A benchmark for long-term memory in chat assistants over many sessions. It is the yardstick Mem0 and similar systems report against, and a concrete picture of what "remember across sessions" is being measured on.

Takeaways

  • The model is stateless between calls, so durable memory is something your code stores outside the window and re-injects a relevant slice of. The store is not the window.
  • A memory system is four operations: extract atomic facts from a turn, embed and store them, retrieve the top-k relevant ones for a new query, and invalidate the stale fact when a new one contradicts it.
  • Retrieval is what keeps the window small. Injecting the few relevant facts (16 tokens in the demo) instead of the whole history (47, and climbing every turn) is a real and growing saving, because top-k scales with the question, not the conversation length.
  • Invalidation needs a key (the subject and attribute of a fact, minus its value) so a new value replaces the old one in place instead of sitting beside it. Without it, retrieval returns contradictory facts at random.
  • Mem0 is the vector-first layer you bolt on; Letta gives the agent an OS-style memory hierarchy it pages itself; Zep stores a temporal graph for when "what was true then" matters. Anthropic's memory tool lets the model read and write a directory you back with storage.

👉 The store we built throws away the past: update the favorite color and blue is gone. But plenty of questions are about the past, and answering "what was the budget last quarter" needs a memory that records when each fact was true. The next chapter builds exactly that: a temporal knowledge graph, where facts carry validity windows and nothing is ever silently overwritten.