What is a noise schedule in a diffusion model?

A noise schedule is a predetermined sequence of Gaussian noise variances used over the timesteps of a diffusion model. By selecting how quickly the data's signal decays, it determines which noisy versions of the training data the denoising model sees and how signal-to-noise ratio (SNR) changes across time.

1. What do beta, alpha, alpha-bar and SNR represent?

• Beta, βₜ: New Gaussian-noise variance added at timestep t, usually with 0 < βₜ < 1.
• Alpha, αₜ: The per-step squared signal-retention coefficient, αₜ = 1 − βₜ.
• Cumulative alpha, ᾱₜ: The product ᾱₜ = α₁ × α₂ × … × αₜ starting from ᾱ₀ = 1.
• Accumulated noise variance: 1 − ᾱₜ, conditional on a fixed original observation x₀.
• Coefficient SNR: SNRₜ = ᾱₜ / (1−ᾱₜ); high SNR means original signal dominates, while low SNR means accumulated noise dominates.

Important assumption: The familiar SNR formula describes the relative *coefficients* of signal and unit-variance Gaussian noise. To interpret it as a ratio of signal and noise powers, clean data should have approximately unit variance. For data variance V, the effective ratio of conditional signal power to noise power is V×ᾱₜ/(1−ᾱₜ).

The forward noising mechanism itself, including direct sampling from x₀, was explained in Question 286. This article is about how to choose the sequence of variances, not how to train the denoising predictor.

2. Linear beta schedule

One approach increases beta by a constant increment between a chosen start and end value:

βₜ = β_start + (t−1)×(β_end−β_start)/(T−1) for t=1,…,T.

• Advantages: Easy to implement, debug and reproduce. A linear beta schedule was used in the original DDPM formulation of Ho et al. (2020).
• What isn't linear: Although β increases linearly, cumulative ᾱₜ is a product of successive 1−βₜ. Therefore cumulative signal and SNR generally do not change linearly.
• Possible issue: Nichol and Dhariwal found that their tested linear baseline produced extremely noisy images near the end of diffusion at certain resolutions; some late states contributed less useful training information.

3. Cosine schedule

Nichol and Dhariwal (2021) instead designed the desired *cumulative alpha-bar curve*:

f(t) = cos²(((t/T + s)/(1+s)) × π/2)

ᾱ_target(t) = f(t)/f(0).

They used the small offset s = 0.008. Convert that target curve into per-step beta values:

βₜ = min(1 − ᾱ_target(t)/ᾱ_target(t−1), 0.999).

• Purpose: Redistribute noising steps across the trajectory so useful signal survives longer in some settings, instead of concentrating many training steps at nearly pure-noise levels.
• Endpoint: The ideal cosine curve reaches zero cumulative signal at t=T. Without a cap, this would imply β_T=1, which creates singular numerical terms; a cap such as 0.999 avoids that exact endpoint.
• Subtlety: Once capped, the actual ᾱₜ must be calculated as the cumulative product of αₜ=1−βₜ. The last value will differ from the uncapped theoretical target.
• Not universally better: Gains depend on the model, number of steps, noise-level range, loss weighting and dataset.

4. Numerical example — four timesteps

For easy arithmetic, deliberately use T=4 and unusually large noise increments. These are *teaching values*, not a production recommendation.

The linear schedule increases beta by 0.15 at each step, using β=[0.25, 0.40, 0.55, 0.70]. The cosine schedule follows the original cosine formula with T=4, s=0.008 and a 0.999 cap.

For each step calculate:

• αₜ = 1−βₜ.
• ᾱₜ = ᾱ₍ₜ₋₁₎ × αₜ, starting from ᾱ₀=1.
• SNRₜ = ᾱₜ/(1−ᾱₜ).

Schedule A — Linear

First step worked out: β₁=0.25, α₁=0.75, ᾱ₁=1×0.75=0.75; hence SNR₁=0.75/0.25=3.

Second step worked out: β₂=0.40, α₂=0.60, ᾱ₂=0.75×0.60=0.45, giving SNR₂=0.45/0.55≈0.8182.

All four steps, reporting beta, alpha, cumulative alpha-bar and SNR:

• t=1: β=0.250000, α=0.750000, ᾱ=0.750000, SNR=3.000000.
• t=2: β=0.400000, α=0.600000, ᾱ=0.450000, SNR=0.818182.
• t=3: β=0.550000, α=0.450000, ᾱ=0.202500, SNR=0.253918.
• t=4: β=0.700000, α=0.300000, ᾱ=0.060750, SNR=0.064679.

Observation: Even though beta increments are equal, SNR falls from 3.0000 → 0.8182 → 0.2539 → 0.06468. It is not a linear decline.

Schedule B — Cosine

Calculate ᾱ_target(t) with the cosine formula, derive each beta using the ratio of successive target values, cap beta at 0.999, then recompute actual alpha-bar.

First step: β₁≈0.152988; α₁≈0.847012; ᾱ₁≈0.847012, giving SNR₁≈5.536467.

Second step: β₂≈0.416958, α₂≈0.583042; hence ᾱ₂≈0.847012×0.583042≈0.493844 and SNR₂≈0.975674.

All four steps:

• t=1: β=0.152988, α=0.847012, ᾱ=0.847012, SNR=5.536467.
• t=2: β=0.416958, α=0.583042, ᾱ=0.493844, SNR=0.975674.
• t=3: β=0.707859, α=0.292141, ᾱ=0.144272, SNR=0.168596.
• t=4: β=0.999000, α=0.001000, ᾱ=0.000144, SNR=0.000144 (the beta value is capped at 0.999).

Important last step: β₄=0.999 makes α₄=0.001; therefore actual ᾱ₄≈0.144272×0.001=0.000144272 and SNR₄≈0.000144293. This is not exactly zero because the endpoint was capped.

5. What does the comparison mean?

• At t=1: The cosine schedule has a higher SNR (5.5365) than this linear schedule (3.0000), so more original-signal coefficient remains.
• At t=2: Cosine remains slightly higher (0.9757 versus 0.8182).
• At t=3: Cosine becomes noisier (0.1686 versus 0.2539).
• At t=4: The capped cosine schedule is much nearer pure Gaussian noise (SNR≈0.0001443) than the illustrated linear schedule (SNR≈0.06468).

Do not interpret these numbers as image-quality benchmarks. The four-step schedules have different final noise levels, and no generator has been trained. The point is that a schedule determines where easy, intermediate and very difficult training examples appear in time. A fair comparison should also align the intended SNR range and training/evaluation conditions.

6. Practical effects on diffusion training

• Early, high-SNR steps: The image remains recognizable and denoising focuses more on fine local changes.
• Intermediate SNR: The model must infer structure hidden by partial corruption. How many timesteps occupy this range affects training emphasis.
• Late, low-SNR steps: Most original information has disappeared. Too many nearly identical pure-noise states can make some training computation unproductive.
• Terminal prior: Many diffusion generators begin reverse sampling from N(0,I); the forward endpoint should be compatible with that distribution, generally requiring very small terminal alpha-bar.
• Training difficulty and sample quality: The schedule interacts with timestep sampling, objective weighting, data normalization and the denoiser architecture. There is no guarantee that a cosine schedule will outperform every linear configuration.
• Empirical evidence: *Improved DDPM* reported that cosine reduced information destruction too early compared with its tested linear baseline; this depends on that experiment, not solely on the cosine name.

7. Small practical implementation

PYTHON
import numpy as np

T = 4
linear_beta = np.linspace(0.25, 0.70, T)

s = 0.008
t = np.arange(T + 1, dtype=np.float64)
f = np.cos(((t / T + s) / (1 + s)) * np.pi / 2) ** 2
target_bar = f / f[0]
cosine_beta = np.minimum(1 - target_bar[1:] / target_bar[:-1], 0.999)

for name, beta in [("linear", linear_beta), ("cosine", cosine_beta)]:
    assert np.all((beta > 0) & (beta < 1))
    alpha = 1 - beta
    actual_bar = np.cumprod(alpha)
    snr = actual_bar / (1 - actual_bar)
    print(name, "beta:", beta, "alpha:", alpha)
    print("alpha_bar:", actual_bar, "SNR:", snr)

This example uses four steps only to show the arithmetic. Real training must choose validated endpoints and an appropriate timestep count. Use sufficient numerical precision for products and log-SNR values.

8. Mistakes, limits and trade-offs

• Beta is not total noise: One-step βₜ differs from total conditional noise variance 1−ᾱₜ.
• Linear beta is not linear SNR: Inspect alpha-bar and SNR rather than reasoning from beta values alone.
• Cosine endpoint clipping: Do not use β=1 blindly. Recompute actual alpha-bar from the clipped betas, particularly if comparing terminal SNR.
• Indexing errors: Mathematics commonly labels t=1,…,T, while arrays use indices 0,…,T−1. Computing cosine beta requires the initial t=0 target as well.
• SNR singularity/underflow: When alpha-bar is near 1 or 0, direct ratios and products can become numerically unstable; careful precision, clamping and log-SNR methods help.
• Train/inference mismatch: Replacing the noise schedule after training is not a simple independent hyperparameter swap. Any inference respacing must use a compatible noise trajectory.
• Data scale: Actual signal power depends on feature/image normalization; compare schedules with consistent preprocessing.
• Fair validation: Compare fidelity, coverage, held-out performance and computational budget across multiple seeds. A low SNR is not itself a measure of sample quality.
• Many alternatives: Linear and cosine are well-known examples, not the only noise-parameterization choices.

Curriculum scope: Question 286 covered the forward Gaussian corruption mechanism. This lesson focuses on the design and interpretation of schedules. Question 288 covers how a neural network learns to predict noise, and Question 289 compares DDIM and DDPM sampling; those mechanisms are reserved for subsequent lessons.

Key takeaway: A noise schedule decides how noise and original signal trade off over time. Linear schedules specify beta directly; cosine schedules design cumulative alpha-bar and derive beta. Track beta, alpha, alpha-bar, SNR and endpoints together to understand the consequences.

Sources: Ho, Jain & Abbeel, *Denoising Diffusion Probabilistic Models* (NeurIPS 2020); Nichol & Dhariwal, *Improved Denoising Diffusion Probabilistic Models* (ICML 2021); Lin et al., *Common Diffusion Noise Schedules and Sample Steps Are Flawed* (WACV 2024).