What is cross-validation in deep learning?

Cross-validation is a model-evaluation technique that repeatedly trains and evaluates a model on different splits of the available data.

The goal is to get a more reliable estimate of how well the model will generalize to unseen data.

How it works

A common method is k-fold cross-validation. The dataset is divided into k parts, called folds.

For each run:

• One fold is used for validation.
• The remaining folds are used for training.
• The process repeats until every fold has been used for validation once.

For example, with 5-fold cross-validation:

Run 1: folds 2–5 train, fold 1 validates

Run 2: folds 1,3–5 train, fold 2 validates

The final performance is usually summarized across all runs.

Why use cross-validation?

Cross-validation can help when:

• The dataset is relatively small.
• You want a more stable estimate than a single validation split.
• You are comparing models or hyperparameters.

Why is it less common for very large deep learning models?

Deep neural networks can be expensive to train. If a model must be trained from scratch several times, k-fold cross-validation can multiply the training cost significantly.

For large datasets and expensive models, a fixed training/validation/test split is often more practical.

For the basic dataset roles, see “What are training, validation, and test datasets?”.

In short, cross-validation evaluates a model across multiple train-validation splits to produce a more reliable estimate of generalization, but it can be computationally expensive for deep learning.