Regularization
Also: regularisation, regularizer
A family of methods that trade a little training-set fit for better performance on unseen data, for example by adding a penalty on large weights to the loss, injecting noise (dropout, augmentation), or stopping training early.
L_total(θ) = L_data(θ) + λ · Ω(θ)
Example
Adding λ·Σw² with λ = 0.01 to a regression loss makes the optimizer prefer weights like (0.8, 0.5) over (12, −11) when both fit the training points about equally well.