Most examples I have seen about Q learning, are performed in a deterministic world. For example, in the traditional grid world, the agent can finally do the path searching by exploring and exploiting the environment with a reward function without knowing the transition probability function.
$$Q(s,a) = Q(s,a) + a*[ Reward + discount * Max Q(s',a') - Q(s,a)] $$
Now suppose the grid is a stochastic environment, an agent can move up/left/right with 1/3 probability. How can I program the Q learning, does that mean that in calculating the $Max Q(s',a')$,
$$Max Q(s',a') = Max [ P(up)*Q(s',up) , P(left) *Q(s',down) , P(right) * Q(s, right)]?$$
