Language model
Also: LM, n-gram language model
A probability distribution over token sequences. By the chain rule, P(w₁…wₙ) factors into next-token predictions P(wₜ | w₁…wₜ₋₁), so learning to predict the next token is enough to score or generate any text. N-gram models truncate the history; neural models condition on all of it.
P(w₁, …, wₙ) = Πₜ P(wₜ | w₁, …, wₜ₋₁)
Example
In the lesson's 12-sentence corpus, "the" is followed by "cat" 7 times out of 20, so an unsmoothed bigram model sets P(cat | the) = 7/20 = 0.35.