Context & memory

why this matters

Context is scarce working space; memory is a product subsystem that selects what to put there. More history can increase cost and distract the model while making stale assumptions feel authoritative.

coverage

• Working context: window limits, instruction/history/tool budgets.
• Compression: summaries, compaction, lossy state.
• Retrieval memory: store, embed, rank, inject.
• State: session state vs durable preferences/facts.
• Quality: relevance, freshness, contradiction, context rot.
• Privacy: consent, visibility, deletion, sensitive inference.

mental model

Memory is a write policy plus a retrieval policy, both under user control.

lessons

worked example

A summary preserves a decision but loses its reason

An assistant compacts ‘Use region A only until EU residency is required’ into ‘Use region A.’ Later an EU customer arrives.

1. Classify context as current task, durable decisions, source evidence, temporary observations, and sensitive user data.
2. Preserve the decision's condition, date, and source when summarizing. A shorter statement that drops the reversal trigger changes meaning.
3. Retrieve relevant durable records for the current task instead of replaying every old tool result.
4. Give persisted user memory explicit write, correction, deletion, and retention rules; do not silently turn every conversation into a permanent preference.

Design decision: Store compact, attributable facts and decision conditions, then retrieve them under a bounded context policy.

Limit: Summaries are lossy and generated memory can be wrong. Keep source links and allow correction rather than treating a summary as authoritative.

Teaching scenario · traffic, prices, and capacity values are illustrative.

assignment

Produce: A memory schema, context assembly policy, and before/after compaction test.

Paper, a text editor, or a spreadsheet is enough. Model access and paid services are optional.

1. Create a fictional conversation with two decisions, a contradiction, a temporary detail, and a sensitive field.
2. Write a compact summary and test whether it answers the later questions without changing conditions or permissions.
3. Specify consent, update, retrieval, expiry, and deletion; compare a no-memory baseline with the proposed design.

Change a constraint

• The user retracts a remembered preference.
• An untrusted document attempts to write a durable instruction into memory.

Self-review rubric

• Decisions retain conditions and provenance.
• Context and durable memory are distinguished.
• The user can inspect, correct, and delete retained information.

If a criterion is missing, revise that part of the artifact before rating your practice below.

Input allocation: policy 1k + request 1k + history 2k + evidence 8k
                  + tool state 2k = 14k; output reserve = 2k
Memory: {subject, tenant, kind, value, source, createdAt, expiresAt,
         authority, sensitivity, consentBasis, supersedes}
Bad summary: “Use region A.”
Retained decision: “Use A only until EU residency is required;
                  then reevaluate under the placement policy.”
Compaction test: EU customer arrives → must recover conditional decision
Retraction: remove preference + invalidate summaries/retrieval references

cumulative practice

Harbor · build and evaluate a support assistant

Keep one project document across this track. Reconstruct earlier reasoning before opening notes, then revise the same artifact. Use part of the assignment and review budget for this work. If you enter mid-course, sketch the earlier decisions first using the linked chapters.

Add to your project: Harbor's context assembler and safe preference memory.

Recall how input length affects serving and how retrieved evidence is authorized. Add a user response-style preference, a current policy, and an old contradictory memory. Compact the run while preserving operation identity and policy provenance; then retract the preference.

Keep your first attempt and revision together. If your answer differs, identify the assumption or mechanism that explains the difference before rating your recall.

senior signal

Treat memory as data product with consent, provenance, and evaluation—not a bigger prompt buffer.

failure patterns

• Persisting every utterance
• Injecting all memories into every turn
• No user correction/deletion mechanism
• Summarizing away permissions or unresolved decisions

knowledge checks

Start with these quick checks, then explain the closed-book questions in your own words.

1. 01What makes a memory safe to inject?

   A.It is oldB.It is relevant, permitted, fresh enough, and provenance-awareC.It is longD.It was generated by a model

   select an option to check your answer

2. 02Why can more context hurt?

   A.It always changes the tokenizerB.Irrelevant/stale material distracts and raises costC.GPUs reject textD.It removes sampling

   select an option to check your answer

closed-book recall

interview drill

Design persistent user memory for a general assistant that works across health, travel, and work contexts.

• cross-domain
• sensitive facts
• user control
• context budget

references

• Anthropic: effective context engineering for AI agents ↗reviewed 2026-10

Sources offer deeper context. Provider capabilities and API contracts can change; check current official docs before implementing.

notes