What is a mini-batch in neural network training?

A mini-batch is a small subset of the training dataset that is processed together before the neural network updates its weights.

Instead of using the entire dataset for every update, training usually splits the data into many mini-batches.

Simple example

Suppose the training dataset contains:

10,000 examples

and the batch size is:

100

Then one epoch contains approximately:

10,000 ÷ 100 = 100 mini-batches

For each mini-batch, the model performs:

Forward pass → Compute loss → Backpropagation → Weight update

Then it moves to the next mini-batch.

Why use mini-batches?

Mini-batches provide a practical balance between using the whole dataset at once and processing one example at a time.

They offer several benefits:

• Lower memory usage than full-batch training.
• Faster computation through efficient GPU parallelism.
• More frequent weight updates.
• Some gradient noise, which can sometimes help optimization.

Mini-batch vs batch size

A mini-batch is the actual group of training examples being processed.

The batch size is the number of examples inside that mini-batch.

For example:

mini-batch = 64 training examples

batch size = 64

Why is it important?

The choice of mini-batch size affects:

• GPU memory usage.
• Training speed.
• Gradient stability.
• How often the optimizer updates the model.

For a deeper discussion of this setting, see “What is batch size in deep learning?” and the next article “How does batch size affect neural network training?”.

In short, a mini-batch is the group of training examples processed together to calculate a gradient and perform one model update.