3
$\begingroup$

Markov Process : A stochastic process has Markov property if conditional probability distribution of future states of process depends only upon present state and not on the sequence of events that preceded.

Markov Decision Process: A Markov decision process (MDP) is a discrete time stochastic control process. It provides a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.

Asper my understanding Markov Decision Process is just a framework for Markov Process or there is something else I am missing. One more question is it says it as Stochastic control process meaning it is not completely random and Markov Process is completely random . Can someone help me with this

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

MDP is an expansion of MP, and the difference is written in your question, but maybe an illustration will help.

MP is a stone rolling down the road and has a probability at each fork in its path to go left or right.

MDP is when you put someone (agent) on the stone that tries to or has the ability to affect the stone's path and steer it a bit to the left or the right. This means that the probability of the stone actually veering to the left or the right is dependent upon the action that the agent takes.

This answer is weird, but I hope it helps you understand.

Improve this answer
$\endgroup$
3
  • $\begingroup$ You mean to say Markov Process : The way environment is designed and Markov Decision Process is a framework or a sort of guide for an agent in that environment $\endgroup$
    – nithin
    Oct 31, 2018 at 9:23
  • 1
    $\begingroup$ no no! I mean MP is still the process of the stone rolling (both are processes!) the difference is that MDP takes into account the action of an agent in at each point $\endgroup$
    – Cherny
    Oct 31, 2018 at 9:27
  • 1
    $\begingroup$ For example, just to be clear, if you have an MDP which only has 1 action (meaning nothing to choose), it will be precisely an MP $\endgroup$
    – Cherny
    Oct 31, 2018 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.