Stochastic gradient descent
Also: SGD, mini-batch gradient descent, mini-batch SGD
Each update uses the average gradient over a random mini-batch, giving many cheap, noisy steps per pass over the data. The noise can help escape shallow traps; mini-batches of 32–256 are the usual compromise.
θ ← θ − η · (1/|B|) Σ_{i∈B} ∇Lᵢ(θ)
Example
With 50,000 training examples, full-batch descent makes 1 update per pass; SGD with batches of 100 makes 500 noisier but far cheaper updates in the same pass.