What happens when the learning rate is too low?

When the learning rate is too low, the optimizer makes very small parameter updates, causing neural network training to progress extremely slowly.

What happens during an update?

A simplified update rule is:

w_new = w_old - learning_rate × gradient

If the learning rate is very small, even a useful gradient produces only a tiny change in the weight.

What problems can this cause?

A learning rate that is too low can lead to:

• Very slow reduction in training loss.
• Many more training steps or epochs being required.
• Accuracy improving only slightly over time.
• Training appearing stuck even though parameters are still changing.

Simple intuition

Imagine trying to reach the bottom of a valley using extremely tiny steps.

You may be moving in the correct direction, but reaching the bottom can take unnecessarily long.

Can a low learning rate ever be useful?

Yes.

Smaller learning rates are often useful later in training when the model is already near a good solution and needs more precise updates.

This is why many learning-rate schedules gradually reduce the learning rate over time.

How can you recognize a learning rate that is too low?

Common signs include:

• Training loss decreases very slowly.
• The model needs unusually many epochs to improve.
• Gradients exist, but parameter updates are tiny.
• Training remains stable but makes little progress.

How can it be fixed?

Common approaches include:

• Increase the learning rate.
• Use a learning-rate finder or systematic tuning.
• Use a learning-rate schedule.
• Check that small gradients are not actually caused by the vanishing gradient problem.

For the broader effect of this hyperparameter, see “How does the learning rate affect neural network training?”.

In short, a learning rate that is too low makes optimization steps too small, resulting in stable but unnecessarily slow learning and convergence.