Dot product
Also: inner product, scalar product
For vectors a and b of equal length, a·b = Σ aᵢbᵢ, which also equals ‖a‖‖b‖cos θ. It underlies neurons (w·x), attention scores (q·k) and similarity search.
a · b = Σᵢ aᵢ bᵢ = ‖a‖ ‖b‖ cos θ
Example
In attention, a query q = (1, 2, 3) scores a key k = (4, −5, 6) as 1·4 + 2·(−5) + 3·6 = 12; a key orthogonal to q would score 0.