How can you tell whether a deep neural network is underfitting?

A deep neural network is likely underfitting when it performs poorly not only on validation or test data, but also on the training data itself.

This means the model has not learned the underlying patterns well enough.

What should you monitor?

Compare training and validation performance.

A typical underfitting pattern looks like:

Training loss: high

Validation loss: high

and both may improve only slowly or stop improving too early.

Common signs of underfitting

• Training loss remains high.
• Training accuracy remains low.
• Validation performance is also poor.
• The gap between training and validation performance is small because both are weak.
• The model cannot capture obvious patterns in the training data.

Why does underfitting happen?

Common causes include:

• The model is too simple for the problem.
• Training stops too early.
• The learning rate or optimization setup is poor.
• Regularization is too strong.
• The input features or data representation are not informative enough.

How is this different from overfitting?

With overfitting, training performance is strong but validation performance is much worse.

With underfitting, even the training performance is poor.

How can it be reduced?

Possible approaches include:

• Train for more epochs.
• Increase model capacity when appropriate.
• Reduce excessive regularization.
• Improve the optimization settings.
• Improve the quality or representation of the input data.

For the basic concept, see “What is underfitting in deep learning?”.

For the opposite behavior, see “How can you tell whether a deep neural network is overfitting?”.

In short, a deep neural network is underfitting when it cannot perform well even on its training data, showing that the model has not learned the problem sufficiently.