What is a base LLM?

A base LLM is a language model that has completed its main pretraining stage but has not yet been specifically tuned to behave like an instruction-following assistant or chat model.

Its core objective is usually to predict the next token from the text that comes before it.

For example, if you give a base LLM:

The capital of France is

it may continue with:

Paris

But if you give it an instruction such as:

Explain photosynthesis in three bullet points.

a base model may not reliably follow the instruction in the way a chat or instruction-tuned model would. It may instead continue the text in a style similar to patterns it saw during pretraining.

How is a base LLM created?

A base LLM is typically trained on a very large corpus of text using a language-modeling objective such as next-token prediction.

During this stage, the model learns patterns involving:

• Grammar and syntax
• Word and concept relationships
• Facts and associations present in the training data
• Writing styles
• Code patterns
• General statistical structure of language

This training gives the model broad language capability, but it does not automatically make the model good at following user instructions.

What happens after the base model?

A base model can be further trained using post-training techniques such as:

• Instruction tuning, where it learns from instruction-response examples
• Preference or alignment training, where responses are optimized toward desired behavior
• Domain-specific fine-tuning, where the model is adapted to a specialized task or field

After this additional training, the resulting model may become an instruction-tuned LLM or a chat LLM.

Base LLM vs. chat LLM

A base LLM is mainly optimized to model and continue text.

A chat LLM is additionally optimized to respond helpfully to conversational instructions.

The detailed differences are covered separately in “What is the difference between a base model and a chat model?”

In short: a base LLM is the pretrained language model before instruction-following or chat-specific post-training is applied. It provides the general language capability that later tuning builds upon.