I'm studying reinforcement learning from Prof. Andrew Ng's lecture notes. Here I came across fitted value iteration algorithm for continuous state MDP. It's mentioned that in this algorithm, we are approximating the value function $V(s)$, over a finite number of states ($s^{(1)}, s^{(2)}... s^{(m)}$) using supervised learning algorithm like linear regression.
Why we need to sample a finite set of states from continuous states? If we are sampling the continuous states, doesn't it mean descretization of states?
Also what is the intuition behind the sampling of states and then using supervised learning algorithm to get value function? Why can't we simply use following equation for Value function $V(s)$?
$$V(s) = R(s) + {\gamma}\max_{a}\int_{s'}P_{sa}(s')V(s')ds'$$