Backpropagation
Also: backprop, backward pass, reverse-mode differentiation
After a forward pass stores each layer's values, backpropagation walks the layers in reverse, multiplying local derivatives (the chain rule) to get ∂L/∂w for every weight in one pass. An optimizer then uses those gradients.
∂L/∂w = ∂L/∂a · ∂a/∂z · ∂z/∂w
Example
For a = w·x with x = 2, w = 0.5, target y = 2 and L = (a − y)²: a = 1, ∂L/∂a = 2(a − y) = −2, ∂a/∂w = x = 2, so ∂L/∂w = −4 and gradient descent increases w.