16
$\begingroup$

What is the difference between Markov chains and Markov processes?


I'm reading conflicting information: sometimes the definition is based on whether the state space is discrete or continuous, and sometimes it is based on whether the time is discrete of continuous.

Slide 20 of this document:

A Markov process is called a Markov chain if the state space is discrete, i.e. is finite or countable space is discrete, i.e., is finite or countable.

http://www.win.tue.nl/~iadan/que/h3.pdf :

A Markov process is the continuous-time version of a Markov chain.

Or one can use Markov chain and Markov process synonymously, precising whether the time parameter is continuous or discrete as well as whether the state space is continuous or discrete.


Update 2017-03-04: the same question was asked on https://www.quora.com/Can-I-use-the-words-Markov-process-and-Markov-chain-interchangeably

$\endgroup$
  • 1
    $\begingroup$ In my experience, the first definition is wrong. Markov chains are used often in the context of sampling from a posterior distribution (MCMC). These posterior can be defined on a finitnte or continuous state space; so the first definition is probably incorrect. The second one makes more sense. However, I don't think there is much difference between though, since I have often seen the phrase, continuous time Markov chains. $\endgroup$ – Greenparker Oct 24 '16 at 10:34
  • 1
    $\begingroup$ i remember what I learned from text book is markov process is most generic term, and markov chain is time discrete and state discrete special case. $\endgroup$ – Haitao Du Oct 24 '16 at 14:14
8
$\begingroup$

From the preface to the first edition of "Markov Chains and Stochastic Stability" by Meyn and Tweedie:

We deal here with Markov Chains. Despite the initial attempts by Doob and Chung [99,71] to reserve this term for systems evolving on countable spaces with both discrete and continuous time parameters, usage seems to have decreed (see for example Revuz [326]) that Markov chains move in discrete time, on whatever space they wish; and such are the systems we describe here.

Edit: the references cited by my reference are, respectively:

99: J.L. Doob. Stochastic Processes. John Wiley& Sons, New York 1953

71: K.L. Chung. Markov Chains with Stationary Transition Probabilities. Springer-Verlag, Berlin, second edition, 1967.

326: D. Revuz. Markov Chains. North-Holland, Amsterdam, second edition, 1984.

$\endgroup$
6
$\begingroup$

One method of classification of stochastic processes is based on the nature of the time parameter(discrete or continuous) and state space(discrete or continuous). This leads to four categories of stochastic processes.

If the state space of stochastic process is discrete, whether the time parameter is discrete or continuous, the process is usually called a chain.

If a stochastic process possesses Markov property, irrespective of the nature of the time parameter(discrete or continuous) and state space(discrete or continuous), then it is called a Markov process. Hence, we will have four categories of Markov processes.

A continuous time parameter, discrete state space stochastic process possessing Markov property is called a continuous parameter Markov chain ( CTMC ).

A discrete time parameter, discrete state space stochastic process possessing Markov property is called a discrete parameter Markov chain( DTMC ).

Similarly, we can have other two Markov processes.

Update 2017-03-09:

Every independent increment process is a Markov process.

Poisson process having the independent increment property is a Markov process with time parameter continuous and state space discrete.

Brownian motion process having the independent increment property is a Markov process with continuous time parameter and continuous state space process.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.