What is the difference between input tokens and output tokens?

Input tokens are the tokens sent to an LLM as part of a request, while output tokens are the tokens the model generates in its response.

Both are created and counted by the model's tokenizer.

What are input tokens?

Input tokens come from the information provided to the model before generation starts.

Depending on the application, they can include:

• System instructions
• User prompts
• Previous conversation turns
• Retrieved documents
• Tool results
• Special formatting or chat tokens

For example:

User prompt → tokenized → input tokens

These tokens form the context the model uses when deciding what to generate next.

What are output tokens?

Output tokens are the new tokens generated by the LLM.

An autoregressive LLM produces them one at a time:

input context
   ↓
output token 1
   ↓
output token 2
   ↓
output token 3
   ↓
...

Each generated token becomes part of the context used to predict the following token.

How do they relate to the context window?

In many LLM systems, input and output tokens share the model's total context capacity.

Conceptually:

input tokens + generated output tokens
≤ available context capacity

The exact limits and accounting rules depend on the model and API.

A very large input can therefore leave less room for the model's response.

Do input and output tokens cost the same?

Not necessarily.

Many hosted LLM APIs price input and output tokens separately, and output tokens may have a different price from input tokens.

The exact pricing depends on the provider and model.

Simple example

Suppose a request contains:

Input:  2,000 tokens
Output:   500 tokens

The model processed 2,000 tokens of supplied context and generated 500 new tokens.

The request therefore involved both input processing and output generation.

Are tokens the same as words?

No.

A token may be:

• A complete word
• Part of a word
• Punctuation
• Whitespace
• A character or byte-based piece

So 1,000 words do not necessarily equal 1,000 tokens.

For the underlying unit, see “What is a token in an LLM?”

In short: input tokens are the tokenized information supplied to an LLM, while output tokens are the new tokens generated by the model. Both affect context usage, computation, and often API cost.