# Introduction to Markov Process (Part 2): Dynamical system and Markov relationship

In my previous post asked here Introduction to Markov process: How to prove that a process is Markov? Part 1, -- a process is Markovian if it follows the memoryless property. Consider, a dynamical system $x_n = f(x_{n-1})$ and a (stationary) Markov chain ${(X_n)}_{n \in \mathbb{Z}}$ in discrete time with each $X_n$ taking its values in a finite set $E$. The canonical space of the Markov chain is the product set $E^{\mathbb{Z}}$. The trajectory $X=(\ldots, X_{-1}, X_{0}, X_1, \ldots)$ of the Markov chain is a random variable taking its values in $E^{\mathbb{Z}}$. Let $\mu$ be its invariant distribution.

Question1: How does one show that a Markov Process generates (iid) random variables or is it only random variables which may not be iid?

Question2: In https://en.wikipedia.org/wiki/Subshift_of_finite_type it says that through the shift operator $T$, a shift dynamical system is a Markov Process. So, it means that the shift dynamical system can generate iid random variables. How? A simplified version of explanation as to how a Markov Chain is a dynamical system and vice-versa (how the dynamical system can be a Markov Chain) will be helpful to clear the concept.

• Q1: i.i.d. sequences are Markov chains, many Markov chains are not made of independent random variables (and the links in your comment do not say otherwise). Q2: All Markov chains are not dynamical systems. An example of dynamical system yielding i.i.d. random variables is the shift on $\{0,1\}^\mathbb Z$ with the uniform probability, then the $0$th projections of the iterates of the shift are indeed i.i.d.
– Did
Commented Sep 26, 2015 at 14:56
• Thank you for your answer. How can Markov Chains give iid sequences since the prev state is fed back to obtain the next state? It can generate random variables but where is the proof that they are iid? Can you link a proof in support of your answers esp Q2? Commented Sep 27, 2015 at 21:10
• "the prev(ious) state is fed back to obtain the next state" There might lie a deep misconception about Markov chains. What is the precise definition you are using? Please be specific.
– Did
Commented Sep 27, 2015 at 21:18
• In the answer below, it is mentioned that Markov chains generate dependent rvs. This has confused me, all definitions (stat.yale.edu/~jtc5/251/readings/…) and in basic text books mention that Markov process is iid. I had also asked the same QUestion to the person who has put up the answer below -- whether iid or not. Commented Sep 28, 2015 at 0:08
• Did is correct. You will likely find reliable definitions here: statslab.cam.ac.uk/~rrw1/markov/M.pdf Commented Sep 29, 2015 at 16:59