Batch size

Also: mini-batch, mini-batch size

Larger batches give a more accurate, smoother gradient and use GPUs efficiently but cost more per step; smaller batches give cheaper, noisier steps. Peak learning rate is usually scaled up with batch size.

Example

With 10,000 examples, a batch size of 32 gives 313 updates per epoch (the last batch holds 16), while a batch size of 1,000 gives just 10.

See it in the lesson: Multi-Layer Perceptrons →

Related terms

More in Optimization & Training