Context window
The context window is the maximum amount of text — measured in tokens — that a model can consider at once, covering the system prompt, the conversation so far, tool results, and the reply it is generating.
Everything the model "sees" on a given turn has to fit in the context window. For an agent this fills up fast: the system prompt, every user message, and every tool result accumulate turn after turn. When the window is exhausted the oldest or least relevant content must be dropped or summarised, or the model simply cannot take it in. Managing the window — what to keep, compact, or retrieve on demand — is one of the central engineering problems of building agents.
Key points
- Measured in tokens (roughly ¾ of a word in English); it bounds input + output combined.
- In an agent it is consumed by the system prompt, the running conversation, and tool results.
- When it fills, context must be compacted, summarised, or fetched on demand — see retrieval-augmented generation.
- A larger window helps but is not free: more tokens mean higher cost, latency, and the risk of the model losing focus.
Learn it by building
Way of the Agent teaches AI agent engineering level by level — concepts like this one, then hands-on practice, XP, and the AI Chronicles.
Start learning free →