Maximum likelihood estimation
Also: MLE, maximum likelihood, log-likelihood, negative log-likelihood, NLL
θ̂ = argmax Σᵢ log P(xᵢ | θ), assuming independent data. Minimizing the negative log-likelihood is the same thing, which is why common losses (squared error, cross-entropy) are MLE in disguise.
θ̂ = argmax_θ Σᵢ log P(xᵢ | θ)
Example
A coin lands heads 7 times in 10 flips. The likelihood p⁷(1 − p)³ is largest at p̂ = 0.7, the observed frequency.