Value function
Also: state-value function, action-value function, V(s), Q(s, a), Q-function, Q-values
The state-value V^π(s) is the expected discounted return starting in s and following π; the action-value Q^π(s, a) first takes action a. Value functions let an agent compare states and actions, and a policy can be read off as the action with the highest Q.
V^π(s) = E_π[ Σₖ γᵏ r_{t+k+1} | S_t = s ]
Example
If from some cell the agent reaches the +10 goal in 3 steps with no other rewards and γ = 0.9, then V(cell) = 0.9² · 10 = 8.1 (the reward arrives on the third transition).