The open-source landscape
TL;DR. This is the reference map: every lever you built by hand in this book, lined up against the real production project that implements it, plus a symptom-driven decision guide (name the problem, then the lever, then the tool). The families are complements, not a menu of alternatives; a working system stacks several at once. See A professional workflow for them combined end to end.
Contents.
You have now built a small version of every lever. This chapter is the map from each lever to the real project that implements it at production scale, so that when you have a problem you can reach for the right tool instead of reinventing it. The aim is to leave you able to do two things: name the technique a project belongs to, and say when you would choose it.
A note on links, in the spirit of the rest of this site: the field moves fast and URLs rot, so most entries name the project and, where the address is one I am confident about, give a plain GitHub org or root domain. For the newer or more niche tools, the name is enough to search. Nothing here is an endorsement; pick by fit and by the quote you get today.
The map, by family
Compression: shrink what is sent and written
What unites this family: every tool here makes the token sequence smaller without changing the task. Compression acts on the content (drop low-information words, select only the code that matters), output reduction acts on the answer (write less), and both are the first thing you reach for when something does not fit or the bill is dominated by sheer volume. None of them needs a database or any persistent state; they are stateless transforms you put in front of the model.
| Lever | Project(s) | What it does | Reach for it when |
|---|---|---|---|
| Prompt and context compression (Ch 3) | LLMLingua and LLMLingua-2 (Microsoft, github.com/microsoft/LLMLingua); plus Claude Code companions RTK (github.com/rtk-ai/rtk) and Headroom (github.com/chopratejas/headroom) | Prune low-information tokens from a long prompt before sending, coarse-to-fine, keeping the meaning | A long RAG context or heavy few-shot block is blowing the budget |
| Tool-output compression (CLI/agent, Ch 3) | RTK (github.com/rtk-ai/rtk) compresses command output; lean-ctx (github.com/yvgude/lean-ctx) compresses CLI/file reads via an MCP server; Headroom (github.com/chopratejas/headroom) is a context-compression layer that can delegate to lean-ctx | Shrink git, find, and test output at the source, before it lands in the window and is re-sent every turn | A coding agent's window fills with verbose command output (logs, diffs, search results) |
| Output token reduction (Ch 4) | Provider params (Anthropic effort, structured outputs, stop sequences); output shapers like caveman and Headroom's shaper | Make the model write less: terser answers, schema-only fields, hard caps | Output volume dominates cost, or you only need a label or a JSON field |
| Code and structure-aware context (Ch 5) | CodeCompressor; lean-ctx (github.com/yvgude/lean-ctx); tree-sitter; Aider's repo map | Select code by AST or call graph instead of dumping whole files | A coding agent or repo QA system is pasting far more code than the task needs |
Caching: stop paying twice
What unites this family: every tool here avoids recomputing work you have already done. Where compression makes the tokens fewer, caching makes the same tokens cheaper the second time you send them, whether that is the KV computation for a stable prefix (Ch 6), the whole answer to a repeated question (Ch 7), or memory pages shared inside the serving engine (Ch 8). Reach for caching when the cost problem is repetition, not size. The catch is stability: a cache only pays off when the thing you cache stops changing, which is why prefix order matters (see the Remember note below).
| Lever | Project(s) | What it does | Reach for it when |
|---|---|---|---|
| KV-cache and prefix caching (Ch 6) | Provider-native prompt caching (Anthropic and others); Headroom's CacheAligner (github.com/chopratejas/headroom) orders the prefix to keep the cache warm | Reuse the cached computation and charge for a stable prompt prefix across calls | A large system prompt or document is re-sent on every call in a session |
| Semantic and response caching (Ch 7) | GPTCache (github.com/zilliztech/GPTCache); Redis LangCache (redis.io) | Return a stored answer for an approximately-similar query, skipping inference | Traffic has many near-duplicate questions (FAQ, support, repeated analytics) |
| KV-cache serving optimization (Ch 8) | vLLM PagedAttention (github.com/vllm-project/vllm); SGLang RadixAttention (github.com/sgl-project/sglang) | Page KV memory and share prefix blocks across concurrent requests in the engine | You run your own inference server and need throughput and memory efficiency |
Memory and state
What unites this family: every tool here moves information out of the live context window and brings back only the relevant slice on demand. Compression and caching work on the tokens of a single call; memory works across calls and sessions, so the agent is not limited to what fits in one window. Reach for it when the problem is durability (it forgets) rather than size or cost: facts that must survive a restart (Ch 9), facts that change over time (Ch 10), old turns that must be summarized rather than dropped (Ch 11), or mistakes that must be turned into a rule the agent will not repeat (Ch 12).
| Lever | Project(s) | What it does | Reach for it when |
|---|---|---|---|
| Agent memory and persistence (Ch 9) | Mem0 (github.com/mem0ai/mem0); Letta (formerly MemGPT, github.com/letta-ai/letta); Zep (github.com/getzep/zep) | Extract, store, retrieve, and invalidate facts across turns and sessions | The assistant must remember users and prior context across sessions |
| Temporal knowledge graphs (Ch 10) | Graphiti (github.com/getzep/graphiti); Zep | Store facts with validity windows so you can ask what was true at a past time | Facts change over time and point-in-time correctness matters |
| Context-window compaction (Ch 11) | Letta tiered memory; provider compaction and context editing (Anthropic) | Summarize or clear old context when the window fills, keeping the gist | Long agent runs or multi-hour chats that overflow the window |
| Failure and procedural learning (Ch 12) | LangMem (github.com/langchain-ai/langmem); headroom learn (github.com/chopratejas/headroom) mines failed sessions and proposes rule edits | Mine past sessions to rewrite the agent's instructions (CLAUDE.md, AGENTS.md) | The agent keeps repeating the same avoidable mistakes |
Orchestration and architecture
What unites this family: the other three families each give you one lever; this family decides which levers to pull, in what order, on each turn, and rests on the model architecture that makes long windows affordable in the first place. Orchestration (Ch 13) is the control layer that assembles the right context and routes the work; attention efficiency (Ch 14) is the underlying property of the model you chose. Reach here when the problem is routing (it pulls the wrong things) or capacity at the model level (you need a genuinely long window).
| Lever | Project(s) | What it does | Reach for it when |
|---|---|---|---|
| Context orchestration (Ch 13) | LangGraph (github.com/langchain-ai/langgraph); lean-ctx (github.com/yvgude/lean-ctx) for lean per-turn assembly | Assemble the right context, tools, and state per turn via a graph with branching and state | A multi-tool agent needs to decide which sources to pull on each turn |
| Long-context attention efficiency (Ch 14) | DeepSeek sparse attention (DSA) and MLA (github.com/deepseek-ai); MiniMax lightning attention (github.com/MiniMax-AI) | Cut attention compute (sparse/linear) or KV memory (low-rank latent) so long windows are feasible | You are choosing or serving a model for very long context, or want to understand why 1M windows are possible |
Don't be confused. The rows are not alternatives to each other; they are complements. A serious system uses several at once: prefix caching (Ch 6) on the stable preamble, compression (Ch 3) on the retrieved docs, a memory store (Ch 9) for cross-session facts, compaction (Ch 11) when the chat runs long, and an orchestrator (Ch 13) deciding which to apply this turn. The question is rarely "which one"; it is "which combination, in what order".
Remember. Diagnose by symptom first, then reach for the lever, then reach for the tool. Do not start from a project you like and go looking for a problem; start from the behavior you see ("it does not fit", "the bill is too high", "it forgets") and let the decision guide below pick the family. The tool is the last step, not the first.
A decision guide
When a context problem shows up, name the symptom first, then the lever:
- "It does not fit." Capacity problem. Compress the biggest part (Ch 3, Ch 5), or move state out to memory and retrieve only the relevant slice (Ch 9), or summarize the old turns (Ch 11).
- "It is too expensive." Cost problem. If the input is large and stable, cache the prefix (Ch 6). If the answer repeats, cache the answer (Ch 7). If the output is verbose, shape it (Ch 4). Remember output is 5x input (Ch 2).
- "It forgets." Durability problem. Add a memory store (Ch 9); if the facts change over time, make it temporal (Ch 10).
- "It repeats the same mistake." Learning problem. Mine the failures and rewrite the instructions (Ch 12).
- "It is slow at high load." Serving problem. Use a paged, prefix-sharing engine (Ch 8), and pick a model whose attention is efficient at your context length (Ch 14).
- "It pulls the wrong things." Routing problem. Put an orchestrator in front that decides per turn what to assemble (Ch 13).
Build or buy
The from-scratch versions in this book are for understanding, not for production. A real semantic cache needs a vector index, eviction, and persistence; a real memory layer needs durability, concurrency, and access control; a real compressor needs a tuned scorer. The projects above have solved those parts. Build the toy to know what the tool is doing, then use the tool. The one place where "build" often wins is the orchestrator (Ch 13): the routing policy is specific to your application, and a hundred lines of your own control flow is frequently clearer than bending a framework to fit.
The purpose of this map is to make that choice cheap. Once you can name the lever a tool belongs to, you can compare it against the from-scratch version you already understand and ask one question: is the hard part (the part you would have to get right yourself) the generic infrastructure or the policy specific to your app? If it is the infrastructure, buy it. If it is the policy, write it. For the Claude Code companion tools (RTK, Headroom, lean-ctx), the build-or-buy call is already made for you: they are a shell hook or an MCP server you register once, and they save tokens on every command and read from then on, so there is rarely a reason to reimplement them. To see all of these levers and tools combined in a single end-to-end setup, follow A professional workflow, which wires prefix caching, the compression hooks, a memory MCP, subagent delegation, and procedural learning into one working loop.
The lesson worth carrying out of this book is that the families are complements, not alternatives. No single tool fixes a context problem on its own; real systems stack several, because each family addresses a different pressure (size, cost, durability, routing) and a serious workload feels all four at once.
Further reading
The starting points below are real and stable; treat the rest of the field as something you search by name, since URLs rot fast.
- The three Claude Code companion tools, each a single repository with its own README and
install steps: RTK (
github.com/rtk-ai/rtk, compresses command output), Headroom (github.com/chopratejas/headroom, a context-compression layer withCacheAlignerandheadroom learn), and lean-ctx (github.com/yvgude/lean-ctx, an MCP context server). - The Model Context Protocol servers directory (
github.com/modelcontextprotocol/servers): the hub for MCP servers like lean-ctx that plug context tools into an agent. - The compression and serving anchors: LLMLingua (
github.com/microsoft/LLMLingua), vLLM (github.com/vllm-project/vllm), and SGLang (github.com/sgl-project/sglang). - The memory anchors: Mem0 (
github.com/mem0ai/mem0), Letta (github.com/letta-ai/letta), and Graphiti (github.com/getzep/graphiti). - This book's own References and further reading page, which gathers every project named above, the papers behind them, and a glossary in one place.
- A professional workflow, the capstone that puts the whole map to work.
Takeaways
- Every lever in this book has a production project behind it; the map above is the technique-to-tool lookup.
- The families are complements, not alternatives. Real systems stack caching, compression, memory, and orchestration together.
- Diagnose by symptom: does not fit (compress, externalize, summarize), too expensive (cache prefix or answer, shape output), forgets (memory, temporal), repeats mistakes (procedural learning), slow at load (paged serving, efficient attention), pulls the wrong things (orchestrate).
- Build the toy to understand the tool, then use the tool. The orchestrator is the part most worth writing yourself.
👉 A map tells you which tool; it does not show the tool running. The next chapter takes seven of the most prominent projects end to end, from a real terminal with a Claude Code session open to a measured number. Continue to The open-source tool tour.