How does backpropagation work?

Backpropagation works by calculating gradients from the output layer back toward the earlier layers of a neural network.

The process starts after the forward pass has produced a prediction and a loss value.

Forward pass
    ↓
Prediction
    ↓
Loss
    ↓
Backpropagation
    ↓
Gradients for each trainable parameter

Step 1: Measure the loss

The network first compares its prediction with the target using a loss function.

That loss becomes the starting point for gradient calculation.

See “What is a loss function in deep learning?” for the role of the loss itself.

Step 2: Start from the output

Backpropagation begins at the output side of the network and determines how a small change in the final layer would affect the loss.

Conceptually:

∂Loss / ∂output

Step 3: Apply the chain rule backward

Each earlier layer affected the layers after it, so backpropagation uses the chain rule to propagate gradients backward through these dependencies.

A simplified example is:

x → Layer 1 → Layer 2 → Loss

To determine how a parameter in Layer 1 affects the loss, the gradients along the path are combined:

change in Loss
      ↑
Layer 2
      ↑
Layer 1 parameter

This is repeated until gradients have been calculated for all trainable weights and biases.

Step 4: Store the gradients

After the backward pass, each trainable parameter has a gradient such as:

∂Loss / ∂w

The gradient tells us how sensitive the loss is to that parameter. See “What is a gradient in deep learning?” for a focused explanation.

Step 5: The optimizer updates the parameters

Backpropagation stops after computing the gradients. An optimizer then uses those gradients to update the parameters.

So the distinction is:

• Backpropagation: computes gradients.
• Optimizer: uses those gradients to update parameters.

In short, backpropagation works by starting from the loss and applying the chain rule backward through the network to calculate gradients for each trainable parameter.