Encoder–decoder

Also: encoder-decoder, seq2seq, sequence-to-sequence

The encoder reads the whole input and the decoder generates the output step by step, conditioning on the encoder’s representation, typically through cross-attention. It suits tasks where input and output differ in length or form, such as translation and summarization.

Example

Translating "the cat sleeps" (3 tokens) into "le chat dort" plus an end token: the encoder builds 3 context vectors, and the decoder emits 4 tokens while attending to them.

See it in the lesson: Sequence Models for NLP →

Related terms

More in Neural Architectures