Principal component analysis

Also: PCA, principal components

PCA centers the data and finds the eigenvectors of its covariance matrix (equivalently, the right singular vectors from an SVD). The top k eigenvectors are the principal components, and each explains a share of variance equal to its eigenvalue over the total.

C = (1/n) XᵀX, C vᵢ = λᵢ vᵢ, explained ratio = λᵢ / Σⱼ λⱼ

PCA finds the direction of greatest variance and projects the data onto it

Example

If the eigenvalues of a 5-feature dataset are 6, 2, 1, 0.6 and 0.4, the first two components explain (6 + 2) / 10 = 80% of the variance, and three reach 90%.

See it in the lesson: Principal Component Analysis →

Related terms

More in Classical ML