How does deep learning work?

Deep learning works by passing data through a multi-layer neural network, measuring how good the network's output is, and repeatedly adjusting its learned parameters to improve performance.

At a high level, training follows this loop:

Input data
   ↓
Forward pass through neural network
   ↓
Prediction
   ↓
Compare prediction with target
   ↓
Calculate loss
   ↓
Backpropagation
   ↓
Optimizer updates weights and biases
   ↓
Repeat

1. Data enters the network

The input must first be represented numerically. Depending on the problem, this might be image pixels, audio samples, numerical features, or vector representations of tokens.

The input is passed into the first layer of the neural network.

2. Layers transform the data

Each layer receives values from the previous layer and computes new representations.

A simplified neuron computes:

z = w₁x₁ + w₂x₂ + ... + wₙxₙ + b
a = activation(z)

Here, x represents inputs, w represents weights, b is a bias, and the activation function produces the neuron's output.

Weights and biases are the parameters the network learns during training.

3. The forward pass produces a prediction

Information moves from the input through the network's layers toward the output.

For an image classifier:

Image
  ↓
Neural-network layers
  ↓
Learned representations
  ↓
Class scores/probabilities
  ↓
Predicted class

This computation from input to output is called the forward pass.

4. A loss function measures the error

During supervised training, the prediction is compared with the correct target using a loss function.

For example:

Expected: Cat
Predicted: Dog

        ↓

Loss function
        ↓
Numerical loss value

The appropriate loss depends on the task. Classification commonly uses cross-entropy-based losses, while regression often uses losses such as mean squared error.

5. Backpropagation calculates gradients

The network needs to determine how changing each parameter would affect the loss.

Backpropagation efficiently computes gradients of the loss with respect to the network's parameters by applying the chain rule through the computational graph.

Conceptually:

Loss
  ↓
Compute gradients backward through layers
  ↓
∂Loss/∂weight
  ↓
Information needed to update parameters

Backpropagation calculates gradients; it does not itself decide the parameter-update rule.

6. The optimizer updates the parameters

An optimizer uses the gradients to modify weights and biases.

A simplified gradient-descent update is:

new_weight = old_weight - learning_rate × gradient

The learning rate controls the size of each update.

Common optimizers include stochastic gradient descent (SGD) and Adam.

7. Training repeats over many examples

Training data is commonly processed in mini-batches. After many parameter updates across the dataset, the network can become better at minimizing its training objective.

One complete pass through the training dataset is called an epoch.

Dataset
  ↓
Mini-batch 1 → forward → loss → backward → update
Mini-batch 2 → forward → loss → backward → update
Mini-batch 3 → forward → loss → backward → update
  ...
  ↓
One epoch completed

Training usually continues for multiple epochs while performance is also evaluated on data that is not being used for parameter updates.

8. The trained model performs inference

After training, the model can process new inputs using the parameters it learned.

During ordinary inference, the model generally performs a forward computation but does not run backpropagation or update its weights.

New input
   ↓
Trained network
   ↓
Forward pass
   ↓
Prediction

Why multiple layers help

Deep networks can learn hierarchical or compositional representations. A layer transforms the representation produced by previous layers, allowing later computations to build on earlier ones.

The exact representations depend on the architecture, training objective, and data.

In short, deep learning works by repeatedly making predictions, measuring error, computing gradients with backpropagation, and updating neural-network parameters with an optimizer. After training, the learned parameters are used to make predictions on new data.