What context engineering is

TL;DR. A language model is a pure function: it turns one block of input text (the context) into output, and it keeps nothing between calls. Context engineering is the practice of deciding what goes into that block, in what form, and at what cost, given that the block has a fixed maximum size and you pay for every token of it on every call.

Contents

The model is a pure function of its context
Why this is a discipline: purpose, goals, and benefits
A context is assembled, every single turn
The two pressures, and the four families
What "good" looks like
A lesson learned: the re-send illusion
Further reading
Takeaways

The introduction gave the one-sentence version: context engineering is managing the model's input as a fixed-size, paid resource. This chapter makes that precise, because the precision is what lets you reason about every later technique instead of memorizing them. The payoff of getting the model right at this level is concrete. Once you see the model as a function with one input slot, the dozens of techniques in the rest of the book stop looking like a grab-bag of tricks and start looking like answers to two or three questions you can ask on every call.

The model is a pure function of its context

A chat model feels stateful. You tell it your name, and three messages later it still knows it. But the model itself remembers nothing between calls. What actually happens is that your application re-sends the whole conversation every time, and the model re-reads it from scratch on each call. Formally, a single call is

$$\text{output} = f_\theta(\text{context})$$

where $f_\theta$ is the frozen network and context is one long token sequence. The weights $\theta$ hold what the model learned in training; the context holds everything about this situation. There is no third place. If a fact is not in the weights and not in the context, the model does not have it during this call, full stop.

Unpack each piece, because the precision is the whole point.

A token is the unit the model reads. Your text is chopped into tokens (roughly word-fragments) by a tokenizer before the model sees any of it; Chapter 2 shows the exact splitting. For now, treat a token as "about three quarters of a word" and think of the context as a list of these units.

The weights $\theta$ are a few hundred billion numbers fixed at the end of training. They do not change when you talk to the model. They encode patterns the model absorbed from its training data: grammar, common facts, how to write Python, the shape of an argument. They are the same on your first call and your millionth. Crucially, they do not contain your name, your codebase, today's date, or what you said thirty seconds ago. None of that was in the training data, so none of it is in the weights.

The context is the one place all of that lives. It is the single argument you pass to the function on this call. Anything the model should act on right now (your question, the file it should edit, the result of the search it just ran) has to be sitting in the context as plain tokens, or it is invisible.

The word pure is doing real work in "pure function." A pure function has no memory and no side effects: the same input always gives the same distribution of outputs, and the call leaves nothing behind. The model is pure in exactly this sense. It does not write anything to disk, it does not update $\theta$, and it does not stash your sentence anywhere for next time. Two calls with byte-identical contexts are two independent events that happen to produce the same answer, not one call that "remembered" the other. (Sampling can still pick different words because the output is a probability distribution, but the distribution itself is fixed by the input.)

This is the whole reason context engineering is a discipline and not a footnote. Because the context is the only channel for live information, every design question reduces to a question about it: what is in the context, in what form, at what cost, on this call?

Remember. The model is a pure function of its context. Anything that is not in the weights and not in the context does not exist for that call. Every later technique in this book is a way to get the right things into that one input slot, cheaply.

Don't be confused. "The model remembers" and "my application re-sends the history" describe the same observable behavior but they are not the same mechanism, and the difference is the entire job. The model is stateless. The memory is something your code maintains and re-injects. When the memory grows past the window, the model does not "forget gracefully": your code must decide what to cut, or the call fails. Mistaking the illusion for the mechanism is how people end up surprised by both the bill and the amnesia.

Why this is a discipline: purpose, goals, and benefits

If the context is the model's only input, why is filling it a job worth a book? Because the context is a fixed-size, paid resource, and those two words force trade-offs on every call. "Fixed-size" means the context has a hard maximum (the context window, measured in tokens) past which the call simply fails. "Paid" means you are billed for every token you put in and every token the model writes out, on every single call, so a context that you re-send a hundred times in a session costs you a hundred times. Naive code ignores both limits until it hits them. The first time you hit the window, a long conversation crashes mid-task; the first time you read the bill, a chatty agent has spent ten times what it needed to. Context engineering is the set of habits that keep you off both walls on purpose rather than by accident.

The purpose is to put exactly the right information in front of the model on each call, at the lowest token cost, given a hard size limit. Not the most information (that blows the budget and, past a point, hurts accuracy because the model has to find the signal in the noise). Not the least (then the model is missing what it needs and guesses). The right information, in a form the model can use, for the price you can afford.

The goals are four, and you will meet them again as the "what good looks like" checklist later in this chapter: a context should be sufficient (it contains what the task needs), lean (it contains little else), cheap to repeat (its stable parts are not re-billed on every call), and durable (state that should outlive one call is stored somewhere the window boundary cannot erase).

The benefits of hitting those goals are direct and measurable, which is rare for a soft discipline. Cost drops, often by an order of magnitude, because you stop re-sending and re-generating tokens you did not need (Chapter 2 puts dollar figures on this, and caching in Chapter 6 drops the repeat cost of stable prefixes). Latency drops for the same reason: fewer tokens in and out means a faster response. Accuracy rises, because a lean, relevant context is easier for the model to attend to than a bloated one where the answer is buried. And reliability rises: sessions stop crashing at the window boundary because you decided in advance what to keep instead of letting the call fail.

Three use-cases show why this is not optional in real systems:

Long inputs. A user pastes a 200-page contract, or asks about a large codebase. The document is bigger than the window, so you physically cannot send all of it. You have to choose, compress, or retrieve the relevant slice (Chapter 3, Chapter 5).
Agent loops. An agent that calls tools accumulates a transcript: each tool result is appended to the context, the next call re-sends all of it, and the context grows every step. Left alone, a long task walks straight into the window and the bill climbs with each turn. The agent needs a policy for what to keep (Chapter 11, Chapter 13).
Cross-session state. A user expects an assistant to remember their preferences next week. The model cannot; nothing carries over between calls. Some store outside the model has to hold that state and re-inject the relevant part on demand (Chapter 9, Chapter 10).

Each of these is the same problem wearing different clothes: more information wants to be in the context than fits or than you want to pay for, so something has to give, and a policy has to decide what.

A context is assembled, every single turn

The context is not one thing you write once. It is rebuilt on every call from named parts: a system prompt, the tool definitions, any retrieved documents, the running conversation, the latest tool results, and the user's new message, concatenated in a fixed order. It helps to know what each part is for:

System prompt. The standing instructions: who the model is supposed to be, what it may and may not do, the output format you want. It is the same on every call of a session, so it is the prime candidate for caching.
Tool definitions. The list of functions the model is allowed to call, each with its name, arguments, and a description. Also stable across a session.
Retrieved documents. Text pulled in for this specific question: a file, a chunk of a manual, the rows of a database. This changes turn to turn and is usually the largest part.
Conversation history. Every previous user message and model reply, in order. This is the part that grows without bound as a session runs, which is why it is the first thing any eviction policy has to reason about.
Latest tool results. If the model called a tool last turn, the output comes back as a new chunk of context for this turn.
User message. The new request. Without it there is nothing to answer, so it is pinned.

Two facts about this list drive everything that follows. Build order matters, because the parts that come first and stay identical across calls (system prompt, tool definitions) are exactly the parts a cache can reuse for free, so you put the stable material at the front and the volatile material at the back (this is the whole game in Chapter 6). And total size matters, because the sum of all the parts has to fit inside the window, and when it does not, your code, not the model, has to drop something before the call goes out.

Here is that assembly as runnable code. It builds a realistic turn from parts, estimates each part's size with a crude word-based count (real counts are Chapter 2), and then enforces three different budgets by dropping the oldest evictable parts until the rest fit. A budget here is just a token ceiling you set below or at the window size, so you decide what to cut on your own terms instead of letting the call fail at the hard limit. The system prompt and the user's message are pinned: they never get dropped, because without them the call is meaningless. Everything else is evictable, and the policy walks the evictable parts newest-first, keeping each one while it still fits and dropping the rest. That "newest-first, drop the oldest" rule is the simplest policy there is, and watching it make bad choices is the point.

"""Assemble an LLM context from its parts and enforce a token budget.

Context engineering, at its most basic, is this: every call you BUILD the input
sequence from named parts, MEASURE it against the window, and DECIDE what to drop
when it does not fit. This script does exactly that with a crude word-based token
estimate (good enough to see the mechanics; real token counts come in chapter 2).

Only depends on the Python standard library. Run:  python3 context_assemble.py
"""

from dataclasses import dataclass


def est_tokens(text: str) -> int:
    """Rough token estimate. Real tokenizers split into sub-words, so tokens are
    a bit more numerous than whitespace words; ~1.3 tokens per word is a fair
    rule of thumb for English prose."""
    return round(len(text.split()) * 1.3)


@dataclass
class Part:
    name: str
    text: str
    pinned: bool = False  # pinned parts are never evicted (system prompt, user turn)

    @property
    def tokens(self) -> int:
        return est_tokens(self.text)


def assemble(parts, budget):
    """Pack parts into a token budget. Pinned parts always stay. Among the rest,
    keep the most RECENT (parts later in the list) and drop the oldest first,
    which is the simplest possible eviction policy."""
    pinned = [p for p in parts if p.pinned]
    evictable = [p for p in parts if not p.pinned]

    used = sum(p.tokens for p in pinned)
    kept_recent = []
    # walk newest-to-oldest, keep while it fits
    for p in reversed(evictable):
        if used + p.tokens <= budget:
            kept_recent.append(p)
            used += p.tokens
    kept = pinned + list(reversed(kept_recent))
    dropped = [p for p in evictable if p not in kept_recent]
    return kept, dropped, used


# A realistic turn: stable preamble, a long retrieved doc, growing history, the ask.
parts = [
    Part("system_prompt", "You are a careful coding assistant. " * 20, pinned=True),
    Part("tool_defs", "search(query) read_file(path) write_file(path, text) " * 12),
    Part("retrieved_doc", "The deployment pipeline runs build.sh on every push. " * 60),
    Part("history_turn_1", "User asked about the database schema. Assistant replied. " * 15),
    Part("history_turn_2", "User asked about migrations. Assistant gave the steps. " * 15),
    Part("history_turn_3", "User asked about rollback. Assistant explained. " * 15),
    Part("user_message", "Now: why did the last deploy fail?", pinned=True),
]

total = sum(p.tokens for p in parts)
print("=== The context as assembled, part by part ===")
for p in parts:
    flag = " [pinned]" if p.pinned else ""
    print(f"  {p.name:16s} {p.tokens:5d} tokens{flag}")
print(f"  {'TOTAL':16s} {total:5d} tokens\n")

for budget in (10_000, 600, 300):
    kept, dropped, used = assemble(parts, budget)
    print(f"=== Budget {budget} tokens ===")
    print(f"  fits as-is: {total <= budget}")
    print(f"  kept   ({used} tok): {', '.join(p.name for p in kept)}")
    print(f"  dropped:            {', '.join(p.name for p in dropped) or '(nothing)'}")
    print()

print("Lesson: the SAME parts produce a different context at every budget. Deciding")
print("what survives the squeeze IS context engineering; the rest of the book is")
print("smarter ways to do it than 'drop the oldest'.")

Running it:

=== The context as assembled, part by part ===
  system_prompt      156 tokens [pinned]
  tool_defs           62 tokens
  retrieved_doc      624 tokens
  history_turn_1     156 tokens
  history_turn_2     156 tokens
  history_turn_3     117 tokens
  user_message         9 tokens [pinned]
  TOTAL             1280 tokens

=== Budget 10000 tokens ===
  fits as-is: True
  kept   (1280 tok): system_prompt, user_message, tool_defs, retrieved_doc, history_turn_1, history_turn_2, history_turn_3
  dropped:            (nothing)

=== Budget 600 tokens ===
  fits as-is: False
  kept   (594 tok): system_prompt, user_message, history_turn_1, history_turn_2, history_turn_3
  dropped:            tool_defs, retrieved_doc

=== Budget 300 tokens ===
  fits as-is: False
  kept   (282 tok): system_prompt, user_message, history_turn_3
  dropped:            tool_defs, retrieved_doc, history_turn_1, history_turn_2

Lesson: the SAME parts produce a different context at every budget. Deciding
what survives the squeeze IS context engineering; the rest of the book is
smarter ways to do it than 'drop the oldest'.

Look at what happened across the three budgets. With room to spare (10,000 tokens), everything goes in: the full 1,280 tokens, nothing dropped. At 600 tokens the policy has to shed 686 tokens, and it cuts retrieved_doc (624 tokens) and tool_defs (62 tokens), leaving 594. Read that result closely, because it is the whole lesson in one line. The retrieved_doc was the deployment-pipeline document, and the user's pinned question was "why did the last deploy fail?" The crude policy just threw away the single most relevant piece of context while lovingly preserving three turns of old chat about database schemas, migrations, and rollbacks that have nothing to do with the question. It did this not out of malice but because "drop the oldest" has no idea what the question is. It optimizes for recency, and recency is a terrible proxy for relevance.

At 300 tokens the squeeze is harder still: only the system prompt, the user message, and the single most recent history turn survive, for 282 tokens. The document is gone again, and now most of the conversation is gone too. The same raw material (the same seven parts, byte for byte) produced three completely different contexts, and the model would answer the same question three different ways depending only on which budget happened to be in force. Nothing about the model changed. Everything about what it could see did.

That policy is the subject of the book. "Drop the oldest" is the dumbest possible version, useful here precisely because its failure is so visible. Every later chapter is a smarter answer to the same question:

Instead of dropping the document, compress it so it still fits (Chapter 3).
Instead of dropping old turns, summarize them into a few tokens (Chapter 11).
Instead of re-sending the stable preamble and re-paying for it, cache it (Chapter 6).
Instead of guessing which document is relevant, retrieve and rank by the question (Chapter 9 and Chapter 13).

The two pressures, and the four families

Every technique in this book exists to relieve one of two pressures, and usually it trades a little of one for a lot of the other:

The window is finite. The context has a hard maximum size, and you cannot fit everything into it, so on a large task you must choose what goes in, shrink the parts so more fits, or externalize state to somewhere outside the window and pull back only the slice you need. This is the capacity pressure, and it is a hard wall: cross it and the call fails outright.
Tokens cost money and time. Even when everything fits, you pay per token in and per token out, on every call. Re-sending a system prompt unchanged on the hundredth call of a session, or letting the model write a verbose answer when a terse one would do, is paying for tokens you did not need. This is the cost pressure, and unlike capacity it is a soft wall: nothing crashes, the meter just runs. The next chapter makes it concrete in dollars.

The two pressures pull in tension, which is why this is engineering and not a recipe. Relieving capacity often costs you something: summarizing old turns to make room (Chapter 11) spends tokens running the summarizer and risks dropping a detail you later needed. Relieving cost often costs you capacity flexibility: caching a prefix (Chapter 6) only pays off if you keep that prefix byte-identical, which constrains how freely you can reorder the context. Almost every choice in the book is a trade of a little of one pressure for a lot of the other, and knowing which pressure you are actually short on is half of picking the right technique.

The four families of techniques map onto these pressures:

Family	What it does	Mainly relieves
Compression (3to5)	makes each part smaller without losing what matters	capacity and cost
Caching (6to8)	reuses work already done on stable or repeated context	cost
Memory (9to12)	stores state outside the window and re-injects only the relevant slice	capacity
Architecture (13to14)	assembles the right context per turn, and makes long windows tractable at all	both

What "good" looks like

A well-engineered context has four properties, and you can audit any system against them. These are the goals from earlier in the chapter, now stated as a checklist:

Sufficient. Everything the task needs is present, in a form the model can use. The relevant document is in the context, not sitting in a database the model cannot see. The instruction is phrased so the model can act on it. A context that is missing what the task needs makes the model guess, and a guessing model is the most expensive kind of wrong: confident and plausible.
Lean. Nothing the task does not need is present. Every token earns its place. Lean is not just about cost; a context padded with irrelevant history or boilerplate makes the model work harder to find the signal, and accuracy falls as the noise rises. Sufficient and lean are in tension, and tuning between them is most of the day-to-day work.
Cheap to repeat. The stable parts (system prompt, tool definitions) are cached, not re-paid, across the many calls of a session. A correct but uncached context can be right and still cost ten times what it should.
Durable. State that should outlive a single call is stored outside the model and re-injected on demand, so the system does not get amnesia at the window boundary. Without this, an assistant forgets your name the moment the conversation grows past the window, no matter how good the rest of the context is.

Hold these four in mind as you read. Most production failures are a violation of exactly one of them: a missing document (not sufficient), a bloated preamble (not lean), a re-charged prefix (not cheap to repeat), or a forgotten fact (not durable). When a system misbehaves, asking "which of the four did we break?" is usually faster than reading the code, because it points you straight at the family of techniques that fixes it.

A lesson learned: the re-send illusion

The single most common mistake, and the one that costs the most money before anyone notices, is believing the model remembers. The chat interface is built to encourage exactly this belief: you type, it replies in context, you type again, it stays on topic. It feels like a running conversation with something that has a memory.

It is not. Behind that interface, your application is holding the entire transcript and sending all of it on every turn. Turn ten does not send one new message; it sends turns one through nine plus the new message, and the model reads the whole pile from scratch. The "memory" is your transcript buffer, re-paid in full each call.

Two surprises follow from missing this, and they are the two surprises this chapter opened with. The first is the bill: people assume a long chat costs a little more per turn, when in fact each turn re-sends everything before it, so cost grows with the square of the conversation length, not linearly. A hundred-turn session is not a hundred small calls; it is a hundred increasingly large calls, and the last one alone may dwarf the first fifty. The second is the amnesia: when the transcript finally outgrows the window, the model does not gently forget the least important thing. Your code has to drop something, and if you never wrote that policy, the call simply errors, or a library silently truncates from one end and the model loses the thread for reasons no one can see in the chat log.

The fix is not a trick. It is to stop treating the conversation as the model's memory and start treating it as a resource your code owns and curates: you decide what to keep, what to compress, what to store outside the window, and what to re-pay for. That decision, made deliberately on every call, is context engineering. The rest of the book is the toolkit for making it well.

Takeaways

The model is a pure function of its context; between calls it remembers nothing. Any "memory" is state your code keeps and re-injects.
The context is rebuilt every turn from named parts, and it must fit a fixed window. When it does not, something gets cut, and the policy that chooses is where quality lives.
Two pressures drive everything: finite capacity and per-token cost. The four families (compression, caching, memory, architecture) each relieve one or both.
A good context is sufficient, lean, cheap to repeat, and durable. Most failures are a violation of exactly one of those.

👉 We have been counting tokens with a hand-wave. The next chapter makes the cost real: how tokens are actually counted, why output is the expensive half, and how to put a dollar figure on a context before you ever send it.

Context Engineering in Depth