How do diffusion models work?

Diffusion models work by learning to reverse a gradual noising process. During training, noise is added to real data in many small steps, and the model learns how to remove that noise.

1. Start with real data

The process begins with a real example, such as an image.

Real image

2. Add noise step by step

Small amounts of random noise are added repeatedly until the original structure is almost completely destroyed.

Real image → Slightly noisy → Noisier → Nearly pure noise

This is called the forward diffusion process.

3. Train the model to predict the noise

During training, the model receives a noisy sample and information about the current noise level.

It learns to predict the noise that was added, or an equivalent target used to reverse the corruption process.

The prediction error becomes the training loss, and backpropagation updates the model parameters.

4. Generate new data by reversing the process

After training, generation begins from random noise rather than a real example.

The model repeatedly removes a small amount of predicted noise:

Random noise → Less noisy → More structured → Generated sample

After enough denoising steps, the sample can resemble data from the training distribution.

How can the model follow a prompt or condition?

Conditional diffusion models can receive extra information such as text, class labels, or another image.

This conditioning guides the denoising process toward a desired output.

For example:

Random noise + text condition → Repeated denoising → Matching image

Why does generation take multiple steps?

Instead of producing the final output in one operation, a diffusion model gradually refines the sample.

This iterative process can produce high-quality results, but it also makes sampling more computationally expensive than some other generative approaches.

For the basic definition, see “What is a diffusion model?”.

In short, diffusion models learn how to remove noise from corrupted data, then use that learned denoising process to transform random noise into new generated samples.