In my previous post asked here Introduction to Markov process: How to prove that a process is Markov? Part 1, -- a process is Markovian if it follows the memoryless property. Consider, a dynamical system $x_n = f(x_{n-1})$ and a (stationary) Markov chain ${(X_n)}_{n \in \mathbb{Z}}$ in discrete time with each $X_n$ taking its values in a finite set $E$. The canonical space of the Markov chain is the product set $E^{\mathbb{Z}}$. The trajectory $X=(\ldots, X_{-1}, X_{0}, X_1, \ldots)$ of the Markov chain is a random variable taking its values in $E^{\mathbb{Z}}$. Let $\mu$ be its invariant distribution.

Question1: How does one show that a Markov Process generates (iid) random variables or is it only random variables which may not be iid?

Question2: In https://en.wikipedia.org/wiki/Subshift_of_finite_type it says that through the shift operator $T$, a shift dynamical system is a Markov Process. So, it means that the shift dynamical system can generate iid random variables. How? A simplified version of explanation as to how a Markov Chain is a dynamical system and vice-versa (how the dynamical system can be a Markov Chain) will be helpful to clear the concept.

Links to Theorum and proof will be additionally very helpful

  • $\begingroup$ Q1: i.i.d. sequences are Markov chains, many Markov chains are not made of independent random variables (and the links in your comment do not say otherwise). Q2: All Markov chains are not dynamical systems. An example of dynamical system yielding i.i.d. random variables is the shift on $\{0,1\}^\mathbb Z$ with the uniform probability, then the $0$th projections of the iterates of the shift are indeed i.i.d. $\endgroup$ – Did Sep 26 '15 at 14:56
  • $\begingroup$ Thank you for your answer. How can Markov Chains give iid sequences since the prev state is fed back to obtain the next state? It can generate random variables but where is the proof that they are iid? Can you link a proof in support of your answers esp Q2? $\endgroup$ – Ria George Sep 27 '15 at 21:10
  • $\begingroup$ "the prev(ious) state is fed back to obtain the next state" There might lie a deep misconception about Markov chains. What is the precise definition you are using? Please be specific. $\endgroup$ – Did Sep 27 '15 at 21:18
  • $\begingroup$ In the answer below, it is mentioned that Markov chains generate dependent rvs. This has confused me, all definitions (stat.yale.edu/~jtc5/251/readings/…) and in basic text books mention that Markov process is iid. I had also asked the same QUestion to the person who has put up the answer below -- whether iid or not. $\endgroup$ – Ria George Sep 28 '15 at 0:08
  • 1
    $\begingroup$ Did is correct. You will likely find reliable definitions here: statslab.cam.ac.uk/~rrw1/markov/M.pdf $\endgroup$ – conjectures Sep 29 '15 at 16:59

(In answer only to Q1) ... Markov chains usually generate non-independent rv's. That's why it's a "chain": the distribution of the next output is usually dependent on some of the previous outputs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.