What is the forward noising process in diffusion models?

The forward noising process in diffusion models gradually corrupts a clean image or data vector by adding Gaussian noise at a sequence of fixed timesteps. It is a predetermined stochastic process—not a neural network that learns how to add noise. After enough appropriately chosen steps, the noisy data become approximately standard Gaussian noise.

How does forward noising work?

Start with clean data x₀. Each step t takes the previous noisy version x_(t−1), scales it down slightly, and adds a new independent Gaussian-noise vector:

q(x_t | x_(t−1)) = N(x_t ; sqrt(1−β_t) × x_(t−1), β_t I)

Equivalently, to draw one actual sample:

x_t = sqrt(1−β_t) × x_(t−1) + sqrt(β_t) × ε_t, where ε_t ~ N(0,I).

• `x₀`: Original clean sample (an image, for example).
• `xₜ`: Noisy version at step t.
• `βₜ` (beta): The new noise variance at this particular step, between 0 and 1.
• `I`: Identity covariance; the added noise has independent coordinates with equal variance.
• `εₜ`: A newly drawn standard-normal vector, independent of earlier noise draws.
• `sqrt(1−βₜ)`: Shrinks the preceding signal before noise is added.

The forward process is a Markov chain: x_t depends directly on x_(t−1), not on earlier states once the previous state is known.

What do alpha and cumulative alpha-bar mean?

Instead of repeatedly writing 1−βₜ, define:

• One-step alpha: α_t = 1 − β_t.
• Cumulative alpha-bar: ᾱ_t = α₁ × α₂ × ... × α_t, the product of all earlier step alphas.
• Remaining original-sample coefficient: sqrt(ᾱ_t).
• Accumulated noise variance conditional on x₀: 1−ᾱ_t.

Because the independent noise terms are Gaussian, the entire multi-step corruption has a convenient closed-form distribution:

q(x_t | x₀) = N(x_t ; sqrt(ᾱ_t) × x₀, (1−ᾱ_t) I)

So it is possible to sample a noisy version from the original image directly:

x_t = sqrt(ᾱ_t) × x₀ + sqrt(1−ᾱ_t) × ε, where ε ~ N(0,I).

Important distinction: β_t is *one step's* newly added noise variance. 1−ᾱ_t is the *total accumulated conditional noise variance* after all steps through t. The single ε in the direct formula summarizes the combined earlier noises; it is not simply the last step's ε_t.

Beginner-friendly numerical example: Three noisy steps

Imagine clean two-feature data:

x₀ = [1, −1].

We choose the following simple values so the changes are visible in only three steps:

• Step 1: β₁ = 0.10, α₁ = 0.90.
• Step 2: β₂ = 0.20, α₂ = 0.80.
• Step 3: β₃ = 0.30, α₃ = 0.70.

These β values are illustratively large; practical image diffusion normally uses many steps and a designed schedule. Detailed schedule choices are reserved for Question 287.

Step 1 — Add noise to the clean data.

Choose the illustrative Gaussian-noise draw ε₁ = [0,1].

x₁ = sqrt(0.90) × [1,−1] + sqrt(0.10) × [0,1]

• sqrt(0.90) ≈ 0.948683 and sqrt(0.10) ≈ 0.316228.
• x₁ ≈ [0.948683, −0.632456].
• ᾱ₁ = 0.90.

Step 2 — Add different noise to x₁.

Choose ε₂ = [−1,0].

x₂ = sqrt(0.80) × x₁ + sqrt(0.20) × [−1,0]

• sqrt(0.80) ≈ 0.894427 and sqrt(0.20) ≈ 0.447214.
• x₂ ≈ [0.401315, −0.565685].
• ᾱ₂ = 0.90 × 0.80 = 0.72.

Step 3 — Add fresh noise again.

Choose ε₃ = [1,−1].

x₃ = sqrt(0.70) × x₂ + sqrt(0.30) × [1,−1]

• sqrt(0.70) ≈ 0.836660 and sqrt(0.30) ≈ 0.547723.
• x₃ ≈ [0.883486, −1.021009].
• ᾱ₃ = 0.90 × 0.80 × 0.70 = 0.504.
• The accumulated conditional noise variance is 1−ᾱ₃ = 1−0.504 = 0.496.

These are example random draws, not observations from a trained model. An individual coordinate can move up or down and can leave the input's original numeric range; progressive corruption is a distributional phenomenon, not guaranteed monotonic movement of every value.

Directly sample x₃ from x₀ in one calculation

The cumulative formula gives:

x₃ = sqrt(0.504) × [1,−1] + sqrt(0.496) × ε.

Choose a new independent Gaussian-noise draw ε = [0.5,−1].

• sqrt(0.504) ≈ 0.709930.
• sqrt(0.496) ≈ 0.704273.
• Clean-data component: [0.709930, −0.709930].
• Combined noise component: [0.352136, −0.704273].
• Direct result: `x₃ ≈ [1.062066, −1.414202]`.

Why is this different from the step-by-step `[0.883486, −1.021009]`? We deliberately used a different random realization. Both constructions produce samples from the same conditional distribution q(x₃|x₀); they do not have to produce identical vectors. To reproduce one specific sequential sample exactly, use the aggregated noise determined by that sequence rather than drawing a new ε.

This ability to jump straight to any selected timestep is essential for efficient diffusion training.

Why does this help train diffusion models?

• Known clean starting point: A dataset supplies the clean observation x₀.
• Choose any timestep: Randomly select t without having to construct x₁,x₂,...,x_(t−1).
• Generate a training input cheaply: Use the direct formula and a newly sampled Gaussian ε to construct x_t.
• Keep track of the injected noise: The training code knows the noise it sampled. It can use the noisy example and known timestep to define a denoising-learning task.
• Learned reverse process is separate: The forward corruption has no learned parameters in the standard DDPM setup. The neural network learns to undo or model the reverse process; learning noise prediction is the focus of Question 288.

Minimal PyTorch training-input example:

PYTHON
# betas: length-T tensor; x0: [batch, channels, height, width]
alpha = 1.0 - betas
alpha_bar = torch.cumprod(alpha, dim=0)
t = torch.randint(0, len(betas), (x0.shape[0],), device=x0.device)
a_bar_t = alpha_bar[t].view(-1, 1, 1, 1)
eps = torch.randn_like(x0)
x_t = a_bar_t.sqrt() * x0 + (1.0 - a_bar_t).sqrt() * eps

The array indices in the snippet start at 0, whereas the mathematical steps are numbered from 1. This code constructs corrupted training data; it does not perform reverse sampling or train a neural network.

When does the result become approximately Gaussian noise?

For any fixed clean x₀:

q(x_T|x₀) = N(sqrt(ᾱ_T) x₀, (1−ᾱ_T) I).

• If a suitable process makes `ᾱ_T ≈ 0`, its mean approaches zero and covariance approaches I.
• Therefore x_T is distributed approximately as `N(0,I)`, largely independent of the original image.
• In this three-step toy example `ᾱ₃=0.504`, so the terminal state is not close to pure standard Gaussian noise yet.
• Reaching a good near-Gaussian terminal distribution depends on the number of steps and chosen coefficients; the detailed design trade-offs belong to Question 287.

Applications, assumptions and limitations

• Applications: This Gaussian corruption framework supports diffusion-based image, audio, video and continuous-feature generation when representations and preprocessing are appropriate.
• Gaussian assumption: The standard DDPM process uses independent isotropic Gaussian increments with known variance, not arbitrary types of corruption.
• Preprocessing matters: Input scale affects how strongly the noise distorts samples; make the training representation and schedule consistent.
• Information is lost: Noisy x_t generally does not uniquely identify its source x₀. Reversing the process is a learned probabilistic challenge.
• Do not confuse variance levels: β_t describes only the latest increment, whereas 1−ᾱ_t describes the accumulated conditional noise variance.
• No guarantee of high-quality generation: Easy corruption does not imply an accurate learned reverse process.
• Numerical safety: Validate the range 0 < β_t < 1, alpha-bar products and tensor broadcasting; high noise at too few steps can discard useful input detail.

Curriculum boundary: Question 285 finished GAN image-to-image translation. This article introduces the diffusion-model forward process. Question 287 covers noise schedules, Question 288 covers learning to predict noise, and Question 289 covers DDIM versus DDPM sampling. These later topics are intentionally not explained in detail here.

Key takeaway: Forward diffusion is a predefined chain of small Gaussian-noise additions. Its cumulative alpha-bar product gives an exact conditional distribution for jumping from `x₀` to any noisy `x_t`, which makes the training-data construction efficient.

Primary references: Sohl-Dickstein et al., *Deep Unsupervised Learning using Nonequilibrium Thermodynamics* (ICML 2015); Ho, Jain & Abbeel, *Denoising Diffusion Probabilistic Models* (NeurIPS 2020).