GRU
Also: gated recurrent unit
The gated recurrent unit merges the LSTM’s forget and input gates into one update gate that interpolates between the old hidden state and a candidate. It has fewer parameters and often matches LSTM accuracy.
hₜ = (1 − zₜ) ⊙ hₜ₋₁ + zₜ ⊙ h̃ₜ
Example
With 100-dim inputs and a 128-dim state a GRU layer has 3 · (128·228 + 128) = 87,936 parameters, 25% fewer than the matching LSTM.