What is context length in an LLM?

Context length in an LLM is the number of tokens currently included in the sequence the model is processing.

It is measured in tokens and can include the prompt, conversation history, retrieved information, system instructions, and generated tokens that are still part of the active sequence.

Context length vs. context window

These two terms are related but not identical.

• Context length is how many tokens are currently being processed.
• Context window is the maximum number of tokens the model can process at once.

For example:

Context window: 128,000 tokens
Current context length: 20,000 tokens

The current request is using only part of the model's available context capacity.

What contributes to context length?

Depending on the model and application, context length may include:

• System instructions
• User prompts
• Previous conversation turns
• Retrieved documents
• Tool results
• Special tokens
• Generated response tokens

The exact accounting depends on the model and API.

Why does context length matter?

As context length increases:

• More of the context window is consumed
• Memory usage can increase
• Inference can become slower
• Attention computation may increase
• Less room may remain for additional input or output

A longer context can provide more information, but only when that information is relevant and the model can use it effectively.

Is context length measured in words?

No.

Context length is normally measured in tokens.

Because different tokenizers split text differently, the same number of words can produce different context lengths across different LLMs.

For example:

Same paragraph
→ Model A: 500 tokens
→ Model B: 650 tokens

Does context length stay fixed during generation?

No.

During autoregressive generation, each newly generated token becomes part of the sequence used to predict the next token.

So the active context length usually grows as the response is generated.

For the maximum supported capacity, see “What is a context window in an LLM?”

In short: context length is the number of tokens currently being processed by an LLM, while the context window is the maximum number of tokens the model can handle at one time.