Softmax
Also: softmax function
Softmax exponentiates each score and divides by the sum of all the exponentials, so larger scores get exponentially more probability mass. It is the standard output layer for multi-class classification and next-token prediction, and it normalizes attention weights.
softmax(z)ᵢ = e^(zᵢ) / Σⱼ e^(zⱼ)
Example
Logits [2, 1, 0.1] exponentiate to [7.39, 2.72, 1.11], which sum to 11.21, giving probabilities ≈ [0.66, 0.24, 0.10].