Q-learning
Also: tabular Q-learning
After each transition (s, a, r, s′), Q(s, a) moves a step α toward the target r + γ·maxₐ′ Q(s′, a′). It is off-policy: it learns the optimal Q* even while behaving exploratorily, for example with ε-greedy actions.
Q(s, a) ← Q(s, a) + α [ r + γ maxₐ′ Q(s′, a′) − Q(s, a) ]
Example
With Q(s, a) = 0, reward r = 1, best next value 2, α = 0.1 and γ = 0.9: Q ← 0 + 0.1 · (1 + 0.9 · 2 − 0) = 0.28.