What is next-token prediction in LLMs?

Next-token prediction is the task of predicting which token is most likely to come next given the tokens that came before it.

For example, given:

The capital of France is

an LLM may assign high probability to:

Paris

and lower probabilities to other possible tokens.

How does it work?

For the current context, the model produces a probability distribution over its entire vocabulary.

Conceptually:

Context:
The capital of France is

Next-token probabilities:
Paris      0.93
Lyon       0.02
London     0.01
...

The model then selects a token according to the decoding strategy being used.

After selecting Paris, the sequence becomes:

The capital of France is Paris

The model then predicts the next token again.

This repeated process allows an LLM to generate complete sentences, paragraphs, code, and conversations.

Why is next-token prediction important?

For many autoregressive LLMs, next-token prediction is also the core training objective.

During training, the model repeatedly receives a sequence of tokens and tries to predict the correct next token.

For example:

Input:
Machine learning is

Target:
useful

If the model gives a low probability to the correct target token, the training process adjusts its parameters to reduce that error.

By performing this task across very large datasets, the model learns patterns involving grammar, facts, syntax, style, code, and relationships between concepts.

Next-token prediction vs. text generation

They are closely related but not identical.

Next-token prediction is the individual prediction step.

Text generation is the repeated application of that step:

predict token → append token → predict again → append again → ...

For the detailed mechanics behind the probability calculation, see “How does an LLM predict the next token?”

In short: next-token prediction is the process of estimating which token should follow the current context. Repeating this prediction step is what allows autoregressive LLMs to generate text.