Transformer
Introduced in 2017, the Transformer drops recurrence entirely: each block applies multi-head self-attention and a position-wise MLP, each wrapped in a residual connection and layer normalization. All positions are processed in parallel, which made training on huge datasets practical.
Example
The original base Transformer had 6 encoder and 6 decoder blocks, d_model = 512 and 8 heads, about 65M parameters; GPT models keep only the decoder stack, BERT only the encoder.