What does the width of a neural network mean?

The width of a neural network refers to how many neurons or units are present in a layer.

A wider layer contains more neurons, giving that layer more capacity to represent different patterns at the same stage of the network.

Simple example

Consider two hidden layers:

Hidden layer A: 32 neurons

Hidden layer B: 256 neurons

Hidden layer B is wider because it contains more neurons.

Why does width matter?

Each neuron can learn a different combination of input features.

Adding more neurons can therefore increase the model's ability to represent complex relationships.

What happens if a network is too narrow?

If a layer has too few neurons, the model may not have enough capacity to capture important patterns.

This can contribute to underfitting.

What happens if a network is very wide?

Increasing width can also increase:

• Number of parameters
• Memory usage
• Computational cost
• Training time
• Risk of overfitting

So wider is not automatically better.

Width vs depth

Width describes how many neurons are in a layer.

Depth describes how many layers the network has.

For example:

More neurons per layer → wider network

More successive layers → deeper network

Both width and depth affect a neural network's capacity, and both are architectural choices that are usually tuned for the task.

For the related concept, see “What does the depth of a neural network mean?”.

In short, network width describes the number of neurons in a layer and influences how much information that layer can represent.