What are hyperparameters in deep learning?

Hyperparameters are settings chosen before or during training that control how a deep learning model is built or trained.

Unlike model parameters such as weights and biases, hyperparameters are not learned directly from the training data through backpropagation.

Common examples

Hyperparameters include:

• Learning rate: controls the size of parameter updates.
• Batch size: determines how many training examples are processed before an update.
• Number of epochs: controls how many times the model sees the training dataset.
• Number of layers and neurons: define the model architecture.
• Dropout rate: controls how much dropout is applied during training.
• Optimizer choice: such as SGD or Adam.
• Weight decay or regularization strength: controls some forms of regularization.

For related concepts, see “What is a learning rate?”, “What is batch size in deep learning?”, and “What are epochs in deep learning?”.

Parameters vs hyperparameters

A simple distinction is:

• Parameters: learned by the model, such as weights and biases.
• Hyperparameters: chosen by the developer or tuning process, such as learning rate, batch size, and number of layers.

Why do hyperparameters matter?

Hyperparameters can strongly affect:

• Training speed
• Stability
• Model capacity
• Generalization
• Final performance

Poor choices can make training slow, unstable, or ineffective even when the model and dataset are otherwise suitable.

The process of finding good settings is called hyperparameter tuning.

In short, hyperparameters are configuration choices that control how a deep learning model is structured and trained, while model parameters are learned automatically from data.