# When is Markov chain a generator for iid sequences

Definition: Markov chain as : A stochastic process $X_1, X_2, \ldots,$ is a Markov process ( or Markov chain) is for any discrete time index $n = 1,2,\ldots,$ , $Pr(X_{n+1} = x_{n+1}| X_n = x_n, \ldots, X_1 = x_1) = Pr(X_{n+1} = x_{n+1}| X_n = x_n)$.

Q1: What is the meaning of the above statement in English?

Q2: Is Markov chain composed of an independent and identically (iid) random variables. Or when the Markov chain is stationary, then we can say that it is iid random variables?

Thank you

• Can you clarify what you mean by " A stochastic process is a generator of iid sequences"? Or add a reference? – Juho Kokkala Nov 4 '15 at 17:59
• I have updated the Question; it was not properly presented earlier. Sorry – Srishti M Nov 4 '15 at 18:33

## 1 Answer

Q1: The above statement means that the probability of a random variable X being equal to some value x at time n + 1, given all the x values that came before it in the sequence, is equal to the probability of X being equal to some value x at time n + 1 given just the value of x that came before it. In other words, X at time n + 1 is only dependent on x at time n, not any other value of x. So in a sequence, you can say that X at time n + 1 is independent of all other x except X at time n.

Q2: By the answer to Q1, all values in the Markov Chain are not independent of each other because $P(X_{n+1} | X_n) \ne P(X_{n+1})$. After enough iterations, the chain (usually) converges in distribution so they would be identically distributed.

That's all I know.

• I don't think your answer to Q3 is correct, because it's not hard to construct Markov chains ($X_i$) in which, say, $X_4$ and $X_2$ are perfectly correlated. Am I misunderstanding your assertion? – whuber Nov 4 '15 at 18:55
• Thank you for your reply, but could you provide a link to some resource where the rational for answer 3 is presented formally. – Srishti M Nov 5 '15 at 0:22