What happens inside a hidden layer of a neural network?

Inside a hidden layer, each neuron receives values from the previous layer, combines them using weights and a bias, and then usually applies an activation function.

This transforms the incoming representation into a new one that can be more useful for the task.

Step 1: Receive inputs

A hidden neuron receives activations from the previous layer:

x1, x2, x3, ...

Step 2: Compute a weighted sum

The neuron calculates:

z = w1x1 + w2x2 + ... + b

This value is called the neuron's pre-activation value.

Step 3: Apply an activation function

The pre-activation is passed through an activation function:

a = f(z)

For example, ReLU may convert negative values to zero while keeping positive values.

Step 4: Pass the result forward

The activation becomes input to neurons in the next layer.

A hidden layer therefore performs a transformation like:

Previous representation → Weighted transformation → Activation → New representation

What does the hidden layer learn?

During training, the network adjusts the weights and biases so hidden neurons respond to useful patterns in the data.

Different layers can learn different levels of representation.

For example, in image recognition:

Pixels → Edges → Textures → Shapes → Object features

The exact meaning of a hidden neuron's output is learned from data rather than manually programmed.

For the broader purpose of hidden layers, see “Why do neural networks need hidden layers?”.

In short, a hidden layer takes the previous layer's outputs, applies learned weighted transformations and nonlinear activation functions, and produces a new representation for the next layer.