Entropy
Also: Shannon entropy, H(X)
H(X) = −Σ p(x) log₂ p(x), the expected value of −log₂ p(x). It is highest for a uniform distribution and zero when one outcome is certain; decision trees split to reduce it.
H(X) = −Σₓ p(x) log₂ p(x)
Example
A fair coin has H = 1 bit. A uniform choice among 5 outcomes has log₂ 5 ≈ 2.32 bits, and a one-hot distribution has 0 bits.