Context window management

The context window is the finite number of tokens the model can attend to in one request — system prompt, conversation history, retrieved documents, tool results, and the model's own output all share it. Everything you include competes for that budget and adds cost and latency.

Good management means being selective: retrieve and include only the passages relevant to the task, summarize or truncate old history, and drop stale tool output. "Stuff the whole knowledge base into the prompt" is an anti-pattern — it raises cost, slows the call, and can bury the signal the model needs.

Management techniques include RAG (fetch just the relevant chunks), summarization/compaction of long histories, and ordering content so the most important material is clearly placed. Larger windows raise the ceiling but do not remove the need to curate — more tokens still cost more and can dilute attention.