Gradient descent
Also: GD
An iterative optimizer: compute the gradient of the loss with respect to the parameters and move θ ← θ − η∇L(θ), where the learning rate η sets the step size. Repeated until the loss stops improving.
θ ← θ − η ∇L(θ)
Example
For L(x) = x² starting at x = 4 with η = 0.25: the gradient is 2x = 8, so x becomes 4 − 0.25·8 = 2, then 1, then 0.5, halving toward the minimum at 0.