Loss function
Also: cost function, objective function, objective
Maps predictions and targets to one number to minimize, such as mean squared error for regression or cross-entropy for classification. Its choice defines what "good" means and shapes the gradients the model learns from.
L(θ) = (1/n) Σᵢ ℓ(f_θ(xᵢ), yᵢ)
Example
Predicting 2.5 for a true value of 3 costs (3 − 2.5)² = 0.25 under squared error and |3 − 2.5| = 0.5 under absolute error; an outlier off by 10 costs 100 versus 10.