How does a Large Language Model work?

A Large Language Model (LLM) works by converting text into tokens, processing those tokens through a neural network—usually a Transformer—and predicting which token should come next.

At a high level, the process looks like this:

1. Text is tokenized

The input is split into tokens. A token may represent a word, part of a word, punctuation, or another text unit.

1. Tokens are converted into numerical representations

Each token is mapped to a vector called an embedding so the model can process it mathematically.

1. The Transformer processes the context

Transformer layers examine relationships between tokens using mechanisms such as self-attention. This helps the model determine which earlier parts of the input are most relevant to the current prediction.

1. The model produces scores for possible next tokens

After processing the context, the model outputs a score for every token in its vocabulary. These scores are converted into probabilities.

1. A next token is selected

The model chooses one token based on those probabilities and the decoding strategy being used.

1. The process repeats

The newly generated token becomes part of the context, and the model predicts another token. Repeating this step produces sentences, paragraphs, code, or other text.

For example:

Input:
The capital of Japan is

Possible next-token probabilities:
Tokyo   0.91
Osaka   0.03
Kyoto   0.02
...

The model may select Tokyo, append it to the input, and then predict the following token.

How does the LLM learn to do this?

During training, the model sees enormous amounts of text and repeatedly tries to predict missing or next tokens, depending on the training objective. Its internal parameters are adjusted when its predictions are wrong.

Over many training examples, those parameters learn statistical patterns involving grammar, facts, writing styles, code structures, and relationships between concepts.

The model does not search a database for each word it produces. During normal generation, it uses the patterns encoded in its learned parameters together with the text currently in its context.

In short: an LLM works by turning text into tokens, processing their relationships through a Transformer, predicting a probability distribution over possible next tokens, selecting one, and repeating the process to generate text.