Context Engineering And Agent Memory

The prevailing mental model is that a larger context window means you can stop thinking about context. Put the documents in, put the history in, put the tool results in, and let the model sort it out. Every team that has operated a long-running agent has discovered the same thing: capacity is not the constraint that bites. Attention, relevance and cost are.

A model attends unevenly across a long context. Material in the middle of a large body of text gets less reliable treatment than material at either end. Contradictory content does not get reconciled, it gets picked from. Stale instructions from thirty steps ago compete with current ones, and the model has no principled way to know which is live. Meanwhile every token in the window is paid for on every subsequent call, so a context that accumulates without bound produces a cost curve that grows quadratically over the course of a run while the useful information in it grows hardly at all.

So the discipline is not "fit everything in". It is deciding, for each call, what the model needs to see in order to make this specific decision well, and deliberately excluding the rest. Context engineering is a curation problem with a budget attached, and on any non-trivial agent it does more for quality than prompt wording does.

Context is a budget, not a container

Treat every request as having a fixed allowance you are spending. Something has to give way for anything you add. Writing it down in that form changes the design conversation immediately, because it forces an ordering.

A workable allocation for an agent turn, in priority order:

1. System instructions and the current goal      - always present
2. Tool schemas for this task only               - scoped, not global
3. Retrieved facts relevant to the current step  - fetched, not preloaded
4. Recent trajectory in full                     - last few steps verbatim
5. Earlier trajectory, compacted                 - summarised, lossy
6. Persistent memory entries that apply          - selected, not dumped

The two lines that do the most work are the second and third. Scoping the tool set to the task removes a fixed cost from every call and improves selection accuracy at the same time. Fetching facts at the point of need rather than loading a corpus up front means the context contains what this decision requires instead of what the whole task might conceivably require.

The instinct to preload feels safer. It is not. A context stuffed with maybe-relevant material degrades the decision you are about to make, and you pay for it repeatedly.

Working memory and persistent memory are different systems

Conflating these is the source of most confused memory designs. They have different lifetimes, different failure modes and different storage.

Working memory is the current run's context: the goal, the steps taken, the results returned, the intermediate conclusions. It is ephemeral, it is large, it is expensive, and it is the thing under pressure. It dies with the run, and that is correct.

Persistent memory is what survives between runs: user preferences, stable facts about the domain, decisions previously taken, corrections a human made. It is small, it is durable, and it must be explicitly written and explicitly retrieved. It is not "the conversation so far".

Three rules keep this clean.

Nothing enters persistent memory implicitly. An agent that writes down everything it encountered accumulates a store dominated by transient detail and its own mistakes. Writes should be deliberate — a specific tool call with a specific reason — and ideally confirmed by a human or by a downstream success signal.

Persistent memory is retrieved, not appended. Loading the whole store into every run defeats the point and reintroduces the problem you were solving. Select the entries relevant to the current task.

Memory needs eviction and correction. A stored fact that has become false is worse than no fact, because the model will trust it over fresh evidence. Timestamp entries, prefer recency on conflict, and provide a path to delete. A memory store with no delete path is an accumulating liability.

The sharpest version of the risk: an agent that learns a wrong fact and writes it down has created a durable defect that will reappear in unrelated runs, and it will be attributed to the model rather than to the store.

Retrieval is the mechanism, not the memory

Memory for an agent is a retrieval problem, and the engineering that makes retrieval survive production is the same engineering set out in RAG architecture that survives production. Chunking, hybrid search, reranking, freshness and the evaluation of retrieval quality all apply unchanged.

What differs is the trigger. In a question-answering system, retrieval happens once at the start against the user's question. In an agent, retrieval happens repeatedly, against a query the agent formulates for itself, at a point in the task where the relevant question may have drifted a long way from the original goal.

That has two practical consequences. First, retrieval should be a tool the agent calls rather than a preprocessing step you run, because the agent knows what it needs at step eleven and you did not. Second, retrieval quality now compounds: a poor result at step four sends the trajectory somewhere unhelpful, and everything after that is downstream of the mistake. In a single-shot system a poor retrieval produces one weak answer. In a loop it produces a wrong path.

Return retrieved material trimmed to what is usable. A retrieval tool returning twelve full documents has not helped; it has moved the context problem one step later.

Compaction and what it costs you

Eventually a long run exceeds its budget and something must be removed. There are three honest strategies and each loses something specific.

Strategy	Mechanism	What it loses	Suits
Truncation	Drop the oldest turns	Early context, including the original constraints	Short tasks with a stable goal
Summarisation	Replace history with a model-written précis	Detail, exact values, the reason a path was abandoned	Long investigative tasks
Externalisation	Write state to a durable store, keep a pointer	Nothing, if the store is queryable	Anything long-running

Truncation is the most dangerous because it is silent and it removes the beginning, which is where the constraints usually live. An agent that has truncated away "do not modify production data" is an agent that no longer knows that.

Summarisation is the common choice and it has a specific pathology: it preserves what was done and loses why alternatives were rejected. The model re-proposes an approach it already tried and discarded, because the discarding was in the detail that got compressed. If you summarise, structure it — keep the goal verbatim, keep constraints verbatim, keep a list of approaches ruled out and why, and compress only the narrative around them. Never let a model-written summary silently replace a hard constraint with a paraphrase of it.

Externalisation is the best of the three and the most work. The run's plan, findings and completed steps live in a durable store; the context carries a compact index and the agent fetches detail when needed. This makes the run resumable and inspectable as a side effect, which you wanted anyway for production operation.

Sub-agent isolation is a context strategy

The strongest argument for the orchestrator-worker pattern in agent architecture patterns is not specialisation. It is that each worker gets a clean context, does one bounded piece of work, and returns a compact result. The orchestrator accumulates summaries rather than raw detail, which is how a task involving thirty tool calls stays inside a workable budget.

Think of it as scoping. A sub-agent that searches a codebase may read fifty files; the orchestrator needs three paths and a conclusion. Those forty-seven irrelevant files never touch the orchestrator's context, and the sub-agent's context dies with it.

The cost is the interface. Everything the orchestrator learns about that subtask is whatever the worker chose to return, and the worker does not know what the orchestrator will need later. This is lossy delegation with the same failure mode as a badly written handover note, and it is why worker return values should be structured and specified rather than free prose. Say what a worker must return — findings, evidence, confidence, what it could not determine — and validate it.

Use isolation when a subtask generates a large volume of material with a small conclusion. Do not use it when the orchestrator genuinely needs the detail: you have then paid for two contexts and a lossy channel to move information you needed anyway.

Why long-running agents degrade

Watch a long run and you will see quality fall as steps accumulate, in a way that is not about the difficulty of the later steps. Several mechanisms are at work, and they compound.

Signal dilution. The goal was stated once, at the start, and is now surrounded by thousands of tokens of tool output. Its relative weight has fallen. Restating the goal and the constraints on every turn is cheap and effective.

Error accumulation. A wrong conclusion at step six sits in context as established fact and is reasoned from at step twenty. Loops rarely revisit earlier conclusions unless prompted to.

Contradiction without resolution. Two tool results disagree. Both remain in context. The model picks, usually the more recent or the more confidently worded, and does not flag the conflict.

Compaction drift. Each summarisation is lossy, and summarising a summary compounds the loss. After several rounds the working state is a paraphrase of a paraphrase.

Instruction decay. A constraint far from the current turn competes with recent tool output that implies otherwise, and recency tends to win.

The mitigations are structural rather than clever. Restate goal and constraints every turn. Keep a small explicit working-state object — current objective, findings so far, ruled-out approaches, open questions — that is maintained deliberately rather than inferred from the transcript. Cap run length and stop rather than degrade. Prefer several short runs with explicit handover to one very long one, because a handover you designed loses less than a compaction you did not.

The cost and latency consequences

Context is the dominant cost term in most agent systems, and the arithmetic is unforgiving in a way that surprises people. Each turn resends the accumulated context. A run of twenty steps whose context grows steadily does not cost twenty times one call; it costs something closer to the sum of a growing series, which is why a run that takes twice as many steps can cost four times as much.

Latency follows the same shape. Time to first token rises with input size, so late steps in a long run are slower than early ones, and the user experience degrades exactly when they are most invested.

Three levers, in order of effect.

Cut what you send. Trimming verbose tool results and scoping tool schemas per task typically removes more cost than any model substitution, and it improves quality rather than trading against it.

Use caching deliberately. Where a provider supports reuse of a stable prefix, structure your context so the stable parts — system instructions, tool schemas — come first and stay byte-identical. Reordering them per turn silently discards the benefit.

Cap the run. A hard step and spend ceiling per run converts an unbounded tail into a bounded failure you can see and handle. This is the same control argued for in inference cost as an architectural constraint, and it is the difference between a bad week and a bad quarter.

What to do on Monday

Instrument context size per turn on one live agent and plot it across a run. Most teams have never looked at this curve and are surprised by both its slope and where it flattens. That plot tells you whether you have a context problem, and where in the task it starts.

Then audit what is in the context at the widest point. Categorise it: instructions, tool schemas, tool results, retrieved material, history. Anything that is not contributing to the decision being made at that turn is a candidate for removal. Verbose tool results are almost always the largest recoverable block, and trimming them is a change to your tool layer rather than to the agent.

Separate your two memories if they are currently one thing. Working memory is the run's context and should die with the run. Persistent memory is a small, explicitly written, explicitly retrieved, timestamped store with a delete path. Write down what is allowed to enter it and who can remove an entry.

Finally, log every compaction event with its input and output, and restate the goal and constraints on every turn. Those two changes cost an afternoon between them and they remove the two failure modes that are hardest to diagnose after the fact.