What is supervised fine-tuning (SFT) in LLMs?

Supervised fine-tuning (SFT) is a stage where a pretrained LLM is trained on examples that pair an input or instruction with a desired response.

The purpose is to teach the model how to respond in a more useful, task-oriented, and instruction-following way.

What does SFT data look like?

A training example might look like:

Instruction:
Explain photosynthesis in simple words.

Desired response:
Photosynthesis is the process plants use to make food from sunlight...

The model processes the instruction and learns to generate the target response.

Why is it called supervised?

It is called supervised because the training examples include the expected output.

The model is not simply predicting arbitrary text from a large corpus. It is shown what a good response should look like.

How does SFT work?

A simplified flow is:

pretrained model
      ↓
instruction-response examples
      ↓
model predicts response tokens
      ↓
compare with desired response
      ↓
calculate loss
      ↓
update trainable parameters
      ↓
instruction-tuned model

The process is repeated across many examples.

What does SFT teach an LLM?

SFT can improve behaviors such as:

• Following user instructions
• Answering questions directly
• Producing structured outputs
• Summarizing text
• Writing code
• Maintaining a desired tone
• Handling multi-turn conversations
• Following task-specific formats

How is SFT different from pretraining?

During pretraining, the model learns broad language patterns from massive datasets, usually through self-supervised objectives such as next-token prediction.

During SFT, the model learns from curated examples showing how it should behave.

A useful distinction is:

Pretraining:
learn broad language capability

SFT:
learn desired response behavior

Is SFT the same as instruction tuning?

They are closely related.

Instruction tuning is usually performed through supervised fine-tuning on instruction-response examples.

However, SFT is a broader training technique and can also be used for other supervised tasks, such as classification or domain adaptation.

Does SFT require a huge dataset?

Not necessarily.

SFT datasets are usually much smaller than pretraining datasets.

Their quality is especially important because inconsistent or incorrect target responses can directly teach the model undesirable behavior.

Useful SFT data should be:

• Accurate
• Diverse
• Consistent
• Representative of real requests
• Clear about the desired output

Is SFT the final stage of post-training?

Not always.

After SFT, a model may go through additional stages such as preference optimization using methods like RLHF or DPO.

These stages can further shape which responses the model prefers to produce.

In short: supervised fine-tuning trains a pretrained LLM on labeled input-output examples so it learns how to follow instructions and produce responses that better match the desired task, format, and behavior.