Gradient clipping
Also: clipping gradients, clip gradients, gradient norm clipping
Most often by norm: if ‖g‖ exceeds a threshold c, rescale g to length c while keeping its direction. It is the standard fix for exploding gradients in RNNs and Transformers.
g ← g · min(1, c / ‖g‖)
Example
With threshold c = 5, a gradient of norm 50 is scaled by 5/50 = 0.1 before the step; a gradient of norm 3 passes through unchanged.