Convexity
Also: convex function, convex loss
f is convex when the straight line between any two points on its graph never dips below the graph. Gradient descent on a convex loss (with a suitable learning rate) reaches the global minimum; neural-network losses are not convex.
f(t·a + (1 − t)·b) ≤ t·f(a) + (1 − t)·f(b), 0 ≤ t ≤ 1
Example
L(w) = w² is convex, so descent from anywhere ends at w = 0. L(w) = w⁴ − 3w² + w is not: started at w = 2 it settles in the local minimum near w ≈ 1.13 instead of the global one near w ≈ −1.30.