Large language model

Also: LLM, LLMs

A language model scaled up in parameters, data and compute, typically a decoder-only transformer pretrained on trillions of tokens and then instruction-tuned. At scale, next-token prediction yields broad abilities such as translation, coding and reasoning from a prompt alone.

Example

Chinchilla (70B parameters, ~1.4T training tokens, about 20 tokens per parameter) outperformed the 4× larger Gopher (280B) trained with the same compute budget.

See it in the lesson: Scaling Laws →

Related terms

More in Language & LLMs