How can you tell whether a deep neural network is overfitting?

A deep neural network is likely overfitting when it performs very well on the training data but noticeably worse on validation or test data.

The clearest sign is a growing gap between training performance and validation performance.

What should you monitor?

Track both training and validation metrics during training.

For example:

Training loss: 0.50 → 0.30 → 0.15 → 0.08

Validation loss: 0.52 → 0.38 → 0.40 → 0.47

Here, training loss keeps improving while validation loss begins to rise.

That pattern strongly suggests overfitting.

Common signs of overfitting

• Training loss keeps decreasing while validation loss increases.
• Training accuracy becomes very high while validation accuracy stops improving.
• The gap between training and validation performance keeps growing.
• The model performs poorly on unseen test data despite excellent training results.

Why does this happen?

The network begins learning details that are specific to the training set, including noise and accidental patterns.

These details help it perform better on training examples but do not generalize to new data.

How can you confirm it?

Use a separate validation set during training and a test set for final evaluation.

If the model consistently performs much better on training data than on unseen data, overfitting is likely.

What can you do about it?

Common approaches include:

• Early stopping.
• Dropout.
• Weight decay or other regularization.
• Data augmentation.
• More training data.
• Reducing model complexity when appropriate.

For the basic concept, see “What is overfitting in deep learning?”.

For a common visual signal, see “What is a validation loss curve?”.

In short, you can tell a deep neural network is overfitting when training performance keeps improving while validation or test performance stops improving or gets worse.