Local minimum

Also: local minima, local optimum, global minimum

Where the gradient is zero and the loss rises in every direction nearby. Non-convex losses can have many; in large networks most give similarly good loss, and saddle points are the bigger obstacle.

Example

L(w) = w⁴ − 3w² + w has a local minimum at w ≈ 1.13 (L ≈ −1.07) and the global minimum at w ≈ −1.30 (L ≈ −3.51); gradient descent started at w = 2 stops at the local one.

See it in the lesson: Gradient Descent Variants →

Related terms

More in Optimization & Training