Why there is no transition probability in Q-Learning (reinforcement learning)? In reinforcement learning, our goal is to optimize state-value function or action-value function, which are defined as following:
$V^{\pi}_s = \sum p(s'|s,\pi(s))[r(s'|s,\pi(s))+\gamma V^{\pi}(s')]=E_{\pi}[r(s'|s,a)+\gamma V^{\pi}(s')|s_0=s]$
$Q^{\pi}(s,a) = \sum p(s'|s,s)[r(s'|s,a)+\gamma V^{\pi}(s')]=E_{\pi}[r(s'|s,a)+\gamma V^{\pi}(s')|s_0=s,a_0=a]$
However, when we use Q-learning method to get the optimal strategy, the update method is like following:
$Q(S,A) \leftarrow \ Q(S,A) + \alpha [R+\gamma max_a(Q(s',a)) -Q(S,A)]$
My question is:
why in Q-learning there is no transition probability $p(s'|s,a)$. Does it mean we don't need this $p$ when modeling MDP?
 A: Algorithms that don't learn the state-transition probability function are called model-free.  One of the main problems with model-based algorithms is that there are often many states, and a naïve model is quadratic in the number of states.  That imposes a huge data requirement.
Q-learning is model-free.  It does not learn a state-transition probability function.
A: For clarity, I think you should replace $max_a(Q', a)$ with $max_a(Q(S', a))$ as there is only one action-value function, we are just evaluating Q on actions in the next state. This notation also hints at where the $p(s'|s, a)$ lies. 
Intuitively, $p(s'|s, a)$ is a property of the environment. We do not control how it works but simply sample from it. Before we call this update we first have to take an action A while in state S. The process of doing this gives us a reward and sends us to the next state. That next state that you land in is drawn from $p(s'|s, a)$ by it's definition. So in the Q-learning update we essentially assume $p(s'|s, a)$ is 1 because that is where we ended up.
This is ok because it's an iterative method where we are estimating the optimal action-value function without knowing the full dynamics of the environment and more specifically the value of $p(s|s', a)$. If you happen to have a model of the environment that gives you this information you can change the update to include it by simply changing the return to $\gamma p(S'|S, A)max_a(Q(S', a))$.
A: In addition to the above, Q-Learning is a model-free algorithm,that means that our agent just know the states what the environment gives to it. In other words, if an agent selects and performs an action, next state is determined by the environment only and gives to the agent. For that reason, the agent do not think about the state-transition probabilities. 
