What is a context window in an LLM?

A context window is the maximum amount of tokenized information an LLM can consider at one time while processing a request.

It is measured in tokens, not words or characters.

What can fit inside the context window?

Depending on the model and application, the context window can include:

• System instructions
• User messages
• Previous conversation turns
• Retrieved documents
• Tool outputs
• Code or other supplied text
• Tokens generated as part of the current response, depending on the model/API design

Conceptually:

System instructions
+ conversation history
+ current prompt
+ supporting context
+ generated tokens
= tokens competing for the available context

The exact accounting rules vary by model and API.

Why does the context window matter?

An LLM can directly use only information that is available within the context it receives for the current inference request.

If important information is outside that available context, the model cannot directly attend to it unless the application brings that information back into the prompt, for example through retrieval or summarization.

Example

Suppose a model supports a context window of:

128,000 tokens

A request containing instructions, conversation history, and documents must fit within that limit together with whatever token budget is reserved for the model's response.

If the combined content exceeds the supported limit, the application may need to:

• Remove older content
• Truncate documents
• Summarize previous information
• Retrieve only relevant sections
• Split the task into multiple requests

Is a larger context window always better?

Not automatically.

A larger window allows more information to be supplied at once, but long-context processing can require more memory and computation.

Also, simply placing more text into the context does not guarantee that the model will use every part equally well. The quality and relevance of the supplied context still matter.

Context window vs. model knowledge

The context window is different from knowledge stored in the model's learned parameters.

Parameter-based knowledge comes from training.

The context window contains information supplied for the current request.

So:

Model parameters → learned knowledge and patterns
Context window    → information available during the current inference

Does a chat application's visible history equal the context window?

Not necessarily.

An application may show a long conversation history while sending only part of it to the model, or it may summarize older messages before including them.

The actual context is the token sequence supplied to the model for that inference request.

In short: a context window is the maximum token-based working context an LLM can process at once. It determines how much prompt, conversation history, retrieved information, and response content can participate in a single inference request.