Embedding

Also: embedding vector, embedding layer

An embedding layer is a lookup table: item i maps to row i of a trainable matrix. More broadly, an embedding is any vector representation produced by a model in which geometric closeness reflects semantic similarity.

Words become points in a vector space; similar meanings sit close together

Example

A 50,000-token vocabulary with 300-dim embeddings is a 50,000 × 300 table (15M parameters); token id 42 becomes row 42, and "cat" ends up closer to "dog" than to "carburetor".

See it in the lesson: Vector Embeddings Fundamentals →

Related terms

More in Neural Architectures