BERT
Also: Bidirectional Encoder Representations from Transformers
Bidirectional Encoder Representations from Transformers (Google, 2018): a transformer encoder pretrained with masked language modeling, predicting randomly hidden tokens from left and right context. It is fine-tuned with a small task head for classification, NER or extractive question answering.
Example
Given "The cat [MASK] on the mat", BERT uses both "The cat" and "on the mat" to predict "sat"; a left-to-right GPT would see only "The cat".