# Q learning in a stochastic environment

Most examples I have seen about Q learning, are performed in a deterministic world. For example, in the traditional grid world, the agent can finally do the path searching by exploring and exploiting the environment with a reward function without knowing the transition probability function.

$$Q(s,a) = Q(s,a) + a*[ Reward + discount * Max Q(s',a') - Q(s,a)]$$

Now suppose the grid is a stochastic environment, an agent can move up/left/right with 1/3 probability. How can I program the Q learning, does that mean that in calculating the $Max Q(s',a')$,
$$Max Q(s',a') = Max [ P(up)*Q(s',up) , P(left) *Q(s',down) , P(right) * Q(s, right)]?$$

-
 Welcome to the site, @user824624. I edited your question to add formatting and to make the English clearer. Make sure it still says what you want. – gung Aug 15 '12 at 15:02