Hessian

Also: Hessian matrix

For f: ℝⁿ → ℝ, the n×n symmetric matrix of ∂²f/∂xᵢ∂xⱼ. Its eigenvalues reveal the local shape of a loss surface: all positive means a minimum, mixed signs a saddle point.

Hᵢⱼ = ∂²f / ∂xᵢ ∂xⱼ

Example

f(x, y) = x² + 3y² has H = [[2, 0], [0, 6]]: both eigenvalues are positive (a bowl), and the surface curves 3× more steeply along y. For x² − y² the eigenvalues 2 and −2 mark a saddle.

See it in the lesson: Eigenvalues & Eigenvectors in ML →

Related terms

More in Linear Algebra & Calculus