What is batch size in deep learning?

Batch size is the number of training examples processed together before the model performs one parameter update.

For example, if a dataset has 1,000 examples and the batch size is 100:

1,000 examples ÷ 100 per batch = 10 batches per epoch

That means the optimizer performs 10 update steps during one epoch.

Why use batches?

Processing the entire dataset at once can require too much memory. Batches let training work on smaller chunks of data while still making frequent parameter updates.

What happens with different batch sizes?

• Small batch size: uses less memory and produces noisier gradient estimates, but updates happen more frequently.
• Large batch size: uses more memory and produces smoother gradient estimates, but updates happen less frequently.

The best batch size depends on the dataset, model, hardware, and optimizer settings.

For the full-pass concept, see “What are epochs in deep learning?”. The related optimization variants are covered in “What is the difference between batch, stochastic, and mini-batch gradient descent?”.

In short, batch size controls how many training examples are used to compute each parameter update.