Context-window management and compaction

TL;DR. A long conversation cannot keep fitting a fixed window, so your code has to evict something every time the budget is about to break. Chapter 1 took the blunt route, drop the oldest turn, which stays under budget but forgets the fact you stated first. Compaction evicts smarter: it folds the oldest turns into a short running summary, keeps that summary in the window, and moves the raw turns to a cheap out-of-window archive. The result is tiered memory, a small fast in-window tier backed by a large slow archive, paged between like RAM and disk. Compaction is lossy (it keeps the gist, not the bytes), so the one discipline that matters is protecting the load-bearing facts with a focus instruction. Deleting context is a separate, lossless tool for spent tool results; long agent runs use both.

Contents

• The vocabulary, defined
• The mechanism
• When you reach for this
• How the real systems do it
• Using the real tool: commands and before/after proof
• Further reading
• Takeaways

A chat that runs long enough will, eventually, not fit. Every turn re-sends the whole history (Chapter 1), the history only grows, and the window is fixed. So at some point your code has to drop something. Chapter 1 showed the dumbest version of that decision: drop the oldest turn. It keeps you under budget, but it is amnesia by design. The fact that mattered most might be the one you said first.

This chapter does better. Instead of throwing old turns away, we summarize them into a few tokens and keep that summary in the window. The raw turns move to cheap storage out of the window; the gist stays. This is called compaction, and it is what lets a multi-hour agent run or a long support chat stay under budget without forgetting the codename you agreed on in turn 2.

The vocabulary, defined

A handful of terms get used loosely in this area, so here they are pinned down. All of them describe state your code maintains, since the model itself is stateless (Chapter 1).

• Working set: the recent turns you keep verbatim and send to the model on this call. These are in-window: they cost tokens and they fit the budget.
• Archive: older raw turns you keep somewhere cheap (a database, a file) but do not send. Out-of-window. They cost storage, not tokens, and the model never sees them unless you pull one back.
• Eviction: the act of removing a turn from the working set. "Drop the oldest" evicts to the trash. Compaction evicts to the archive, but only after extracting the gist.
• Compaction: replacing a batch of old turns with a short summary of them, so the summary stays in-window while the raw turns leave it.
• Tiered memory: the whole arrangement, a small fast in-window tier (working set plus summary) backed by a large slow out-of-window tier (the archive). The system pages between them the way an operating system pages between RAM and disk.

The tiering is the part worth slowing down on, because it is what separates this chapter from Chapter 1. Think of two tiers with very different properties. The in-window tier is the working set plus the running summary: it is what the model actually reads on this call, so every token in it costs money and counts against the fixed budget. It is fast (the model sees it directly) but small (the window is finite). The out-of-window tier is the archive: the raw turns you set aside in a file or database. It is slow (the model cannot see it without you fetching a row and pasting it back in) but effectively unbounded (storage is cheap and does not count against the window). Compaction is the policy that decides what lives in the small fast tier and what gets paged down to the large slow one, and the summary is the bridge: a few tokens that stand in for many archived turns so the gist stays reachable without the bytes.

Now put the two eviction policies side by side, because the difference is exactly what each one loses. Chapter 1's drop-oldest is eviction to the trash: when the window is about to overflow, it deletes the oldest turn and that turn is gone for good, raw bytes and meaning together. What it loses is everything in that turn, including any decision you made early. Its one virtue is that it is free and stateless. Summarize-on-overflow, this chapter's policy, is eviction to the archive by way of a summary: before the oldest turn leaves the window, its load-bearing facts are folded into the running summary, and only then does the raw turn move to storage. What it loses is the wording and the detail of the old turn, not its conclusions. The trade is plain: drop-oldest spends nothing and forgets the gist; summarize spends a summarization step (and, in production, a model call) to keep the gist. When the early turns carry decisions you will need later, that step pays for itself.

One more distinction matters before the code, because the two operations are easy to blur. Context editing is lossless removal: you take a block that is genuinely irrelevant now, a finished tool result nobody will reference again, and delete it outright. Nothing is summarized because nothing needs to survive; you are just reclaiming the tokens a spent block was holding. Compaction is lossy summarization: you take a block whose conclusions still matter and replace it with a shorter version that keeps the meaning while dropping the exact text. Editing removes what is dead; compaction compresses what is old but still alive. The cost profiles differ too: editing is free of risk because you only delete what you already judged irrelevant, while compaction always risks dropping a detail the summary did not think to keep. That risk is the lesson of this chapter, and the focus instruction (below) is how you manage it.

Don't be confused. Deleting context and summarizing it are different operations with different costs. Deleting (also called context editing) removes blocks outright: you might clear out old tool results that no longer matter, and those bytes are simply gone. Summarizing (compaction) replaces a block with a shorter version that keeps the gist: the raw bytes leave the window but the meaning stays, in fewer tokens. Delete when the content is genuinely spent (a stale tool output nobody will reference again). Summarize when the content is old but its conclusions still matter (the early turns where you settled the requirements). Picking the wrong one is how you either bloat the window with dead tool dumps or forget a decision you still need.

The mechanism

The plan is a loop. Keep appending turns to the working set. After each turn, check whether the in-window total (the summary plus the working set) would blow the budget. While it would, take the oldest working-set turn, fold its key facts into a running summary, and move the raw turn to the archive. Stop when you fit again, or when the working set is down to a small floor of the most recent turns that you always keep verbatim.

Two design choices make this work and not silently lose things:

1. The summarizer keeps facts, not prose. A real system asks the model to write the summary. To keep this demo reproducible (and runnable on a box with no API key, see the provider section below), we use a deterministic stub that pulls out the sentences that look load-bearing: a stated decision (a codename, a deadline, a chosen option) or a sentence that names an entity. Everything chatty is dropped. The exact rule does not matter; the point is that the summary is much smaller than the turns it replaces.
2. The summary itself is bounded. If you only ever append to the summary, it grows without limit and eventually breaks the budget on its own. So the summary is capped: when it is full, the lowest-priority facts fall off first. Decisions outrank passing mentions, so the codename (a decision) is never the fact that gets dropped to make room for small talk.

Here is the whole thing, with a head-to-head against Chapter 1's drop-oldest policy built in. We plant an important fact early ("the project is codenamed Atlas" in turn 2), run a realistic 19-turn conversation that piles up tokens, and then, on the last turn, ask for the codename. The question is answerable only if "Atlas" still appears in what we would send.

"""Context-window management by COMPACTION (summarize, don't just drop).

A long conversation has to keep fitting a fixed window, but you don't want to
lose the facts that were said early on. This script builds a TIERED context
manager that keeps a token budget while preserving the gist of evicted turns:

  - WORKING SET: recent turns kept verbatim, in-window, sent to the model.
  - ARCHIVE:     older raw turns kept out-of-window (cheap storage, not sent).
  - SUMMARY:     a compact running note that captures the key facts (decisions,
                 named entities) from turns that have been pushed out of the
                 working set, so they survive even though the raw text doesn't.

When the in-window total would exceed the budget, we COMPACT: take the oldest
working-set turns, extract their key facts into the running summary, move the
raw turns to the archive, and keep the summary plus the most recent turns.

We compare this against chapter 1's policy ("drop the oldest") by planting an
important fact early and asking about it late. Drop-oldest loses the fact;
compaction keeps it in the summary, under the same budget.

Standard library only. Run:  python3 compaction.py
"""

import re

# Same crude estimate as chapters 1 and 2: ~1.3 tokens per English word.
# Real billing-grade counts come from the provider's count_tokens endpoint.
def est_tokens(text):
    return round(len(text.split()) * 1.3)


# --- The stub summarizer ----------------------------------------------------
# A real system asks the model to summarize. Here we use a deterministic stub
# so the output is reproducible and the mechanism is visible: we keep the lines
# that look load-bearing (a stated decision, or a sentence naming an entity)
# and drop the chatty rest. This is a stand-in for "the model wrote a summary",
# not a serious summarizer.

DECISION_RE = re.compile(
    r"\b(codenamed|named|is called|decided|chose|will use|deadline|budget|"
    r"must|the goal is|launch|ships?|version)\b",
    re.IGNORECASE,
)
# A capitalized word that is not the first word of the sentence: a crude proxy
# for a named entity (a project, person, product, or place).
ENTITY_RE = re.compile(r"(?<!^)(?<![.!?]\s)\b([A-Z][a-zA-Z0-9]{2,})\b")


def key_facts(turn_text):
    """Pull the fact-bearing sentences out of one turn's text, each tagged with
    a priority: 2 if it states a decision (codename, deadline, choice), 1 if it
    only names an entity. Higher-priority facts are kept first when the summary
    itself has to be trimmed, so a decision like the codename outranks chatter."""
    facts = []
    for sentence in re.split(r"(?<=[.!?])\s+", turn_text.strip()):
        s = sentence.strip()
        if not s:
            continue
        if DECISION_RE.search(s):
            facts.append((2, s))
        elif ENTITY_RE.search(s):
            facts.append((1, s))
    return facts


def summarize(turns, prior_summary, max_facts=4):
    """Fold a batch of turns into the running summary, de-duplicating, then
    keep only the top `max_facts` by priority so the summary stays bounded.
    Ties break toward the EARLIER fact, so a fact stated once early survives."""
    facts = list(prior_summary)
    seen = {s for _, s in facts}
    for t in turns:
        for prio, s in key_facts(t["text"]):
            if s not in seen:
                facts.append((prio, s))
                seen.add(s)
    # Stable sort by descending priority; Python's sort is stable, so within a
    # priority the original (chronological) order is preserved.
    facts.sort(key=lambda pf: -pf[0])
    return facts[:max_facts]


def render_summary(facts):
    if not facts:
        return ""
    lines = [s for _, s in facts]
    return "Summary of earlier conversation:\n- " + "\n- ".join(lines)


# --- The tiered context manager --------------------------------------------

class TieredContext:
    """Keeps the in-window context under `budget` tokens by compacting the
    oldest working-set turns into a running summary."""

    def __init__(self, budget, keep_recent=2):
        self.budget = budget          # token ceiling for what we SEND
        self.keep_recent = keep_recent  # min recent turns to keep verbatim
        self.working = []             # recent raw turns (in-window)
        self.archive = []             # older raw turns (out-of-window)
        self.summary = []             # running list of key facts (in-window)
        self.compactions = 0

    def in_window_tokens(self):
        used = est_tokens(render_summary(self.summary))
        used += sum(est_tokens(t["text"]) for t in self.working)
        return used

    def add(self, role, text):
        self.working.append({"role": role, "text": text})
        self._compact_if_needed()

    def _compact_if_needed(self):
        # While we're over budget and still have turns we're allowed to evict,
        # fold the oldest working-set turn into the summary and archive it.
        while self.in_window_tokens() > self.budget and \
                len(self.working) > self.keep_recent:
            oldest = self.working.pop(0)
            self.summary = summarize([oldest], self.summary)
            self.archive.append(oldest)
            self.compactions += 1


# --- Chapter 1's policy, for comparison ------------------------------------

class DropOldest:
    """Naive baseline: keep recent raw turns under budget by discarding the
    oldest. No summary, so anything dropped is gone for good."""

    def __init__(self, budget):
        self.budget = budget
        self.working = []
        self.dropped = 0

    def in_window_tokens(self):
        return sum(est_tokens(t["text"]) for t in self.working)

    def add(self, role, text):
        self.working.append({"role": role, "text": text})
        while self.in_window_tokens() > self.budget and len(self.working) > 1:
            self.working.pop(0)
            self.dropped += 1


# --- Answerability check ----------------------------------------------------
# Late in the chat we ask "what is the project codename?". The answer is
# available only if "Atlas" still appears somewhere in what we'd send: the
# summary or the working-set turns.

def can_answer_codename(visible_text):
    return "Atlas" in visible_text


def visible_text_tiered(ctx):
    parts = [render_summary(ctx.summary)] + [t["text"] for t in ctx.working]
    return "\n".join(parts)


def visible_text_dropoldest(ctx):
    return "\n".join(t["text"] for t in ctx.working)


# --- A scripted ~18-turn conversation --------------------------------------
# Turn 2 plants the load-bearing fact. The rest is realistic filler that piles
# up tokens and pushes the early turns toward eviction.

SCRIPT = [
    ("user",      "Hi, I'm kicking off the planning for our new internal search service."),
    ("assistant", "Great. For the record, the project is codenamed Atlas. I'll refer to it that way from here on."),
    ("user",      "We need it to index about ten million documents to start."),
    ("assistant", "Ten million is fine for a first cut. We can shard the index across a few nodes and grow later."),
    ("user",      "What embedding dimension should we use for the vectors?"),
    ("assistant", "Start at 768 dimensions. It balances recall against memory, and you can revisit it after measuring."),
    ("user",      "How should we handle re-indexing when documents change?"),
    ("assistant", "Use an append-only log of changes and replay it nightly. That keeps the live index stable during the day."),
    ("user",      "Our latency target is under 100 milliseconds at the 95th percentile."),
    ("assistant", "Achievable. Keep the hot shards in memory and cache the most frequent queries to hit that target."),
    ("user",      "Should we expose a REST API or a gRPC one for the search endpoint?"),
    ("assistant", "Offer gRPC internally for speed and a thin REST gateway for external callers who want simplicity."),
    ("user",      "What about access control on the documents?"),
    ("assistant", "Filter results by the caller's permission set at query time, and never index secrets into the vectors."),
    ("user",      "How many engineers do you think this needs for the first quarter?"),
    ("assistant", "Three is a reasonable starting team: one on indexing, one on serving, one on the API and client work."),
    ("user",      "When should we aim to ship the first internal preview?"),
    ("assistant", "Target the end of the quarter for an internal preview, then harden it before any wider rollout."),
    ("user",      "Remind me, what is the project codename again? I need it for the ticket."),
]


def run(strategy_factory, visible_fn, label):
    ctx = strategy_factory()
    print(f"=== {label} ===")
    print(f"  budget: {ctx.budget} tokens, "
          f"turns: {len(SCRIPT)}")
    over = 0
    peak = 0
    for i, (role, text) in enumerate(SCRIPT, start=1):
        ctx.add(role, text)
        used = ctx.in_window_tokens()
        peak = max(peak, used)
        if used > ctx.budget:
            over += 1
    answerable = can_answer_codename(visible_fn(ctx))
    print(f"  peak in-window tokens: {peak}  (budget {ctx.budget})")
    print(f"  turns over budget:     {over} of {len(SCRIPT)}")
    if hasattr(ctx, "compactions"):
        print(f"  compactions:           {ctx.compactions}")
        print(f"  raw turns archived:    {len(ctx.archive)}")
        print(f"  summary facts kept:    {len(ctx.summary)}")
    if hasattr(ctx, "dropped"):
        print(f"  raw turns dropped:     {ctx.dropped}")
    verdict = "ANSWERABLE" if answerable else "UNANSWERABLE (fact lost)"
    print(f"  late codename question: {verdict}")
    print()
    return ctx


BUDGET = 120

print("Planted fact: turn 2 says the project is codenamed Atlas.")
print(f"Late question (turn {len(SCRIPT)}): 'what is the project codename?'\n")

drop = run(lambda: DropOldest(BUDGET), visible_text_dropoldest, "Chapter 1 policy: drop the oldest")
tier = run(lambda: TieredContext(BUDGET), visible_text_tiered, "This chapter: tiered compaction")

print("What the compacting manager would actually SEND on the final turn")
print("(running summary, then the verbatim working set):")
print("-" * 68)
print(visible_text_tiered(tier))
print("-" * 68)
print()
print("Both stayed under the same", BUDGET, "token budget. Drop-oldest lost the")
print("codename when turn 2 fell out of the window; compaction kept it in the")
print("summary, so the late question is still answerable.")

Running it:

Planted fact: turn 2 says the project is codenamed Atlas.
Late question (turn 19): 'what is the project codename?'

=== Chapter 1 policy: drop the oldest ===
  budget: 120 tokens, turns: 19
  peak in-window tokens: 119  (budget 120)
  turns over budget:     0 of 19
  raw turns dropped:     13
  late codename question: UNANSWERABLE (fact lost)

=== This chapter: tiered compaction ===
  budget: 120 tokens, turns: 19
  peak in-window tokens: 118  (budget 120)
  turns over budget:     0 of 19
  compactions:           17
  raw turns archived:    17
  summary facts kept:    4
  late codename question: ANSWERABLE

What the compacting manager would actually SEND on the final turn
(running summary, then the verbatim working set):
--------------------------------------------------------------------
Summary of earlier conversation:
- For the record, the project is codenamed Atlas.
- When should we aim to ship the first internal preview?
- Should we expose a REST API or a gRPC one for the search endpoint?
- Offer gRPC internally for speed and a thin REST gateway for external callers who want simplicity.
Target the end of the quarter for an internal preview, then harden it before any wider rollout.
Remind me, what is the project codename again? I need it for the ticket.
--------------------------------------------------------------------

Both stayed under the same 120 token budget. Drop-oldest lost the
codename when turn 2 fell out of the window; compaction kept it in the
summary, so the late question is still answerable.

Read the two blocks side by side, because the comparison is the point. Both policies held the line on the same 120-token budget: drop-oldest peaked at 119 tokens, compaction at 118, and neither went over on any of the 19 turns. So on the budget metric they are a tie. The difference is what survived. Drop-oldest discarded 13 raw turns into nothing, and turn 2 was one of them, so by the final turn the codename is simply gone and the question cannot be answered. Compaction archived 17 raw turns but distilled them into a 4-fact running summary first, and "Atlas" is the top line of that summary, so the same question is answerable at the same cost.

Notice the summary in the final block sits at four facts and stays there. That is the bound doing its job: as new decisions arrive, the lowest-priority facts age out, but "the project is codenamed Atlas" is a decision, ranks high, and holds its place. Without that cap the summary would have grown every turn and eventually broken the budget itself, which is the trap a naive "just keep summarizing" loop falls into.

That this worked is not free, and here is the lesson the demo is built to teach. Compaction is lossy: the summarizer keeps what it judges load-bearing and throws the rest away, so a fact only survives if the ranking happens to rate it high enough to stay under the cap. "Atlas" survived because the stub scored a stated decision above small talk, but a real summarizer can and will drop a detail it underweights. The fix is not to summarize less; it is to tell the summarizer what to protect. That is what a focus instruction does (you will see it as /compact keep ... later in this chapter, and as the priority ranking in the demo): you name the facts that are load-bearing so the lossy step is steered away from them. The takeaway is to never assume a compactor will keep the one fact you need. Point at it.

Remember. Compaction trades detail for room: it keeps the gist and discards the wording. Because that trade is lossy, the facts you cannot afford to lose are not safe by default. Name them in a focus instruction (or rank them high in your policy) so the summary is steered to preserve them. A summary you did not direct will keep what it thinks matters, which is not always what you need three turns later.

When you reach for this

Compaction earns its complexity on long-running, single-conversation work where the history genuinely outgrows the window:

• Multi-hour chats. Support sessions or pairing sessions that accumulate hundreds of turns. The early turns set context that still matters; you cannot afford to drop them and you cannot afford to keep them all verbatim.
• Tool-heavy agent loops. An agent that calls tools dozens of times fills the window with tool results fast. Here you often want both operations from the "Don't be confused" box: delete the spent tool outputs (context editing) and summarize the reasoning that produced them (compaction).
• Long autonomous runs. An agent working a task overnight will blow any fixed window without some form of paging. Compaction plus an archive is how it keeps going.

For state that has to outlive a single conversation (across sessions, across restarts), this in-conversation summary is not enough; you want the durable, queryable memory of Chapter 9 and Chapter 10. Compaction manages one conversation's window; agent memory manages knowledge across many.

How the real systems do it

The tiered model in the demo is the same one production systems use, with more machinery.

Letta (formerly MemGPT) makes the tiers explicit and lets the agent manage them itself. The MemGPT paper (Packer et al.) framed the whole thing as an operating system for the context window: the window is RAM, external storage is disk, and the agent runs the paging. Letta splits memory into three tiers that map almost one to one onto the demo. Core memory is always in-window: the essential persona and the facts that must never page out, the equivalent of our running summary's top, protected entries. Recall memory is the recent conversation held verbatim, our working set. Archival memory is everything older, stored out of the window and searchable, our archive. The difference from the demo is who decides. In the demo the policy is hard-coded: a fixed rule folds the oldest turn into the summary when the budget is tight. In Letta the agent self-pages: the model is given tools to move data between tiers and it calls them on its own. When the window fills it calls a tool to summarize recall memory down into core; when it needs an old fact it calls a search tool to pull a matching row out of archival memory and back into the window. Self-paging is the same RAM-and-disk picture as the demo, with the model holding the controller instead of a hand-written loop. The cost is a model call per page; the benefit is that the agent decides what is load-bearing in context rather than a static rule guessing for it.

Anthropic's API offers both operations as server-side features, so you do not have to hand-roll the loop. The follow-along below shows the exact calls (the build box has no API key, so the output is illustrative, not a verified block).

Compaction summarizes earlier context into a compaction block as you approach the window. The one rule that bites people: you must append response.content back to your messages each turn, because the compaction state lives in those returned blocks. Strip out just the text and the summary is silently lost.

# Illustrative: requires the anthropic SDK and an API key.
import anthropic

client = anthropic.Anthropic()
messages = []

def chat(user_message):
    messages.append({"role": "user", "content": user_message})
    response = client.beta.messages.create(
        betas=["compact-2026-01-12"],            # the compaction beta header
        model="claude-opus-4-8",
        max_tokens=16000,
        messages=messages,
        context_management={"edits": [{"type": "compact_20260112"}]},
    )
    # Append the FULL content, not just the text: the compaction blocks in here
    # are what the API uses to replace the compacted history next turn.
    messages.append({"role": "assistant", "content": response.content})
    return next(b.text for b in response.content if b.type == "text")

Context editing is the delete side of the "Don't be confused" box: it removes old blocks rather than summarizing them. The clear_tool_uses strategy strips out stale tool results, which is exactly right for a tool-heavy loop where the outputs are spent but the conversation should keep its shape.

# Illustrative: requires the anthropic SDK and an API key.
response = client.beta.messages.create(
    betas=["context-management-2025-06-27"],     # the context-editing beta header
    model="claude-opus-4-8",
    max_tokens=16000,
    messages=messages,
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
    tools=tools,
)

The two are complementary, and a long agent run often uses both: clear the tool results that are done with (clear_tool_uses), and compact the reasoning and decisions that are old but still load-bearing (compact_20260112). Note the different beta headers and edit types: the clearing strategy is not the compaction one, and mixing them up is a common slip.

There is a second payoff that is easy to miss, and it connects back to Chapter 6. Prefix caching reuses the model's work on any stable leading run of tokens that repeats unchanged across turns, so you only pay to process the new tail. A long, ever-growing transcript fights this: the older turns sit far back in the prompt, and although they do not change, the sheer length means more of the request has to be re-read on every call as the tail grows. Worse, summarize-on-overflow that prepends a freshly rewritten summary each turn changes the prefix every turn, which invalidates the cache outright. The fix is to keep the post-compact prefix short and stable: compact at a chosen moment, leave the resulting summary fixed for the rest of the session, and append new turns after it. A short stable prefix (a small summary plus the recent turns, identical on every following call) caches cleanly, so the per-turn cost stays low, not just the per-turn token count. Compaction buys you two things at once here: a smaller window and a more cacheable one.

Using the real tool: commands and before/after proof

The "How the real systems do it" section showed the call shapes. This section runs a real chat loop with them and explains how you prove the compaction is actually working: you watch the input-token count instead of letting it climb forever.

Here is a long chat loop that turns server-side compaction on. The point to get right is the last line of the loop: you append response.content back to messages, not just the text you pulled out of it. The compaction state the API builds up lives inside those returned content blocks. If you keep only the text and throw the rest away, the next request has no summary to stand on and the API has to start the whole job over.

# Follow-along: needs the anthropic SDK and an API key (neither is on this box,
# so the token figures further down are labeled illustrative, not measured here).
import anthropic

client = anthropic.Anthropic()
messages = []

def chat(user_message):
    messages.append({"role": "user", "content": user_message})
    response = client.beta.messages.create(
        betas=["compact-2026-01-12"],                       # turns on server-side compaction
        model="claude-opus-4-8",
        max_tokens=16000,
        messages=messages,
        context_management={"edits": [{"type": "compact_20260112"}]},
    )
    # Append the FULL content, not just the text. The compaction block the API
    # may have written lives in here; strip it and the summary is lost next turn.
    messages.append({"role": "assistant", "content": response.content})
    print("input tokens this turn:", response.usage.input_tokens)
    return next(b.text for b in response.content if b.type == "text")

# Run this dozens of times. As the history grows toward the window, the API
# starts replacing the older turns with a compaction block on its own.
for turn in range(40):
    chat(f"... turn {turn} of a long working session ...")

The delete-side sibling is context editing, which clears spent blocks instead of summarizing them. It uses a different beta header and a different edit type, so do not mix the two up: the clearing strategy is clear_tool_uses_20250919, not the compaction one.

# Follow-along: same caveats. This clears stale tool results rather than summarizing.
response = client.beta.messages.create(
    betas=["context-management-2025-06-27"],                # the context-editing beta header
    model="claude-opus-4-8",
    max_tokens=16000,
    messages=messages,
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
    tools=tools,
)

The before/after proof: bounded input tokens

The metric that tells you compaction is working is input tokens per call, and the test is whether that number stays bounded as the conversation gets long. Every request re-sends the whole history (Chapter 1), so on the naive path, where you just keep appending raw turns, the input-token count on each call climbs in step with the transcript and eventually slams into the window. With compaction on, once the history nears the trigger the API folds the older turns into a compaction block, and the prompt you send on later calls is that block plus the recent turns rather than the full transcript. So input_tokens on a turn-30 call lands far below what the same turn would have cost if you had sent everything.

You read the result off two places in the response:

• response.usage.input_tokens, the count printed in the loop above. On the naive path it rises every turn; with compaction on it rises, then drops back down at the turn where the API emits the summary, then rises again from that lower floor.
• a compaction content block in response.content, which is the summary the API wrote. Its presence is the direct signal that compaction fired on that turn (and it is exactly the block you must append back, per the loop above).

Concretely, with small illustrative numbers (expected shape, not measured on this box):

              naive "send the whole transcript"   with compaction on
turn 1                         ~3k input tokens    ~3k input tokens
turn 10                       ~60k input tokens   ~60k input tokens
turn 30                      ~180k input tokens   ~30k input tokens   <- summary fired

The two paths track each other early, while the history still fits cheaply. They split once the history nears the window: the naive path keeps climbing toward turn-30's ~180k, while the compacted path drops back to ~30k when the API replaces the early turns with a summary block. Same conversation, a fraction of the per-call cost, and the conversation keeps going instead of hitting the wall.

We cannot run that against the live API on this box (no key), so the figures above are the expected shape, not a measured one. The on-box proof is the verified demo earlier in this chapter: it holds the in-window total under a fixed 120-token budget across all 19 turns while keeping the planted "Atlas" fact answerable. That is the same bounded-cost, kept-the-gist result the real tool produces, run end to end with output you can check.

A worked Claude Code session

Claude Code runs this loop for you, so the clearest way to understand compaction is to watch a real session hit the wall and recover. Picture two hours of debugging: you have read a dozen source files, run the failing test several times, and pasted in stack traces and log dumps. The window is filling. To see how full, run /context, which prints a breakdown of what is sitting in the window right now (a companion command, /cost, shows the token spend instead). It looks like this:

> /context

Context window: 170k / 200k tokens used (85%)
  System prompt + tools .......  12k   (7%)
  Files read (14 files) .......  78k  (46%)
  Tool outputs (tests, logs) ..  61k  (36%)
  Conversation ................  19k  (11%)

Eighty-five percent full, and most of it is dead weight: the full text of fourteen files and a pile of old test and log output. You do not need any of that verbatim anymore. What you need to carry forward is small: how to reproduce the bug, which test fails, and the plan for the fix. So you compact with a focus instruction, which tells Claude Code what to protect when it writes the summary:

> /compact keep the repro steps, the failing test name, and the fix plan; drop the file dumps

Compacting conversation... done.
Summarized 41 earlier messages into a summary block.

Claude Code reads the whole history, writes a short summary that honors your instruction, and replaces the old turns with it. Run /context again and the window has dropped sharply:

> /context

Context window: 60k / 200k tokens used (30%)
  System prompt + tools .......  12k   (7%)
  Summary of earlier work .....   4k   (2%)
  Conversation ................  44k  (22%)

From 85% back to 30%, and the facts you marked survived: the repro steps, the failing test, and the fix plan are all in that 4k summary, so the next message continues the work as if nothing was dropped. The file dumps and stale logs are gone, which is the point: they were spent.

You do not always have to ask. As a session approaches the window, Claude Code auto-compacts on its own, folding the older turns down so the conversation keeps going without you pruning anything by hand. Running /compact yourself is for when you want to fold the history at a chosen moment, with a focus instruction, rather than waiting for the automatic trigger to pick for you.

There is a caching payoff too. As Chapter 6 covered, the model caches a stable prefix and reuses it across turns, so it only pays to process the new tail of each request. A long, ever-growing transcript keeps shifting, which limits how much of it caches. After /compact the prefix is short and stable: a small summary plus the recent turns, the same on every following call. That stable prefix caches well, so the per-turn cost stays low for the rest of the session, not just the per-turn token count.

Contrast this with the naive policy from Chapter 1, drop the oldest turn. It would also have pulled the window back under budget, but it would have thrown away those early debugging turns outright, including the repro steps you settled on first. Compaction keeps the gist of those turns in the summary instead. It is the same result as the verified from-scratch demo earlier in this chapter: there, the codename "Atlas" was planted in turn 2 and survived in the running summary under a fixed budget; here, your repro steps were established early and survive the same way. Different content, same mechanism, run for you by the tool.

Further reading

These are the primary sources behind the systems in this chapter. The patterns here are stable, but the exact beta headers and edit-type strings change as the APIs version, so check the current docs before you wire anything up.

• "MemGPT: Towards LLMs as Operating Systems," Packer et al., on arxiv.org. The paper that framed the context window as RAM and external storage as disk, with the agent paging between them. It is where the core/recall/archival tiers and self-paging come from.
• github.com/letta-ai/letta, the open-source successor to MemGPT. Read it for a working implementation of the three tiers and the memory-management tools the agent calls itself.
• Anthropic's compaction documentation on platform.claude.com. The server-side compact_20260112 edit: how it summarizes earlier context, and the rule that you must append response.content back each turn so the compaction state survives.
• Anthropic's context-editing documentation on platform.claude.com. The delete side: clear_tool_uses_20250919 and how it strips spent tool results without summarizing them.
• Claude Code /compact documentation at code.claude.com/docs. The /compact and /context commands, the focus instruction, and how auto-compaction triggers as a session approaches the window.

Takeaways

• A long conversation cannot keep fitting a fixed window, so something gets evicted. "Drop the oldest" stays under budget but is amnesia by design: the most important fact is often the earliest.
• Compaction evicts smarter. It summarizes the oldest working-set turns into a compact running summary, archives the raw turns out of the window, and keeps the gist in-window for a fraction of the tokens.
• Tiered memory is the arrangement: a small in-window tier (working set plus summary) backed by a large out-of-window archive, paged between like RAM and disk.
• Bound the summary, or it becomes the new leak. Cap it and let low-priority facts age out first, so decisions survive and small talk does not.
• Deleting context and summarizing it are different tools. Delete spent tool results (context editing); summarize old-but-still-relevant reasoning (compaction). Long agent runs use both.
• The providers offer both server-side: Anthropic's compact_20260112 summarizes and clear_tool_uses_20250919 deletes; Letta exposes core/recall/archival tiers the agent pages itself. Same tiered idea, less hand-rolling.

👉 Compaction keeps the facts of a long run alive. The next chapter keeps the lessons alive: when an agent fails, how it records what went wrong and turns that into a procedure it follows next time, so the same mistake is not repeated. On to Chapter 12.