What happens to dropout during inference?

During inference, dropout is normally turned off so all neurons remain active when the model makes predictions.

Dropout is mainly a training-time regularization technique.

What happens during training?

During training, dropout randomly sets some neuron outputs to zero.

For example, with:

dropout rate = 0.5

roughly half of the selected activations are dropped on each training pass.

This randomness helps reduce overfitting.

What changes during inference?

During inference:

• No neurons are randomly dropped.
• The full network is used.
• Predictions become deterministic for a fixed input, apart from other stochastic operations.

What about activation scaling?

Most modern frameworks use inverted dropout during training.

With inverted dropout, the activations that remain active are scaled during training so that no additional scaling is needed during inference.

For example, if the keep probability is:

0.5

the surviving activations are typically scaled by:

1 / 0.5 = 2

during training.

This keeps the expected activation magnitude similar between training and inference.

Why is dropout disabled at inference time?

The goal during inference is to use the complete learned model rather than randomly removing parts of it.

Keeping dropout active would normally make predictions vary from one forward pass to another.

Is dropout ever intentionally used during inference?

Yes, in specialized techniques such as Monte Carlo dropout, dropout can remain enabled across multiple inference passes to estimate predictive uncertainty.

However, standard inference disables it.

For the training-time role of dropout, see “How does dropout reduce overfitting in neural networks?”.

In short, dropout is normally disabled during inference, all neurons are used, and modern inverted-dropout implementations require no extra activation scaling at prediction time.