2
$\begingroup$

Definition: Markov chain as : A stochastic process $X_1, X_2, \ldots, $ is a Markov process ( or Markov chain) is for any discrete time index $n = 1,2,\ldots,$ , $ Pr(X_{n+1} = x_{n+1}| X_n = x_n, \ldots, X_1 = x_1) = Pr(X_{n+1} = x_{n+1}| X_n = x_n)$.

Q1: What is the meaning of the above statement in English?

Q2: Is Markov chain composed of an independent and identically (iid) random variables. Or when the Markov chain is stationary, then we can say that it is iid random variables?

Thank you

$\endgroup$
  • $\begingroup$ Can you clarify what you mean by " A stochastic process is a generator of iid sequences"? Or add a reference? $\endgroup$ – Juho Kokkala Nov 4 '15 at 17:59
  • $\begingroup$ I have updated the Question; it was not properly presented earlier. Sorry $\endgroup$ – Srishti M Nov 4 '15 at 18:33
1
$\begingroup$

Q1: The above statement means that the probability of a random variable X being equal to some value x at time n + 1, given all the x values that came before it in the sequence, is equal to the probability of X being equal to some value x at time n + 1 given just the value of x that came before it. In other words, X at time n + 1 is only dependent on x at time n, not any other value of x. So in a sequence, you can say that X at time n + 1 is independent of all other x except X at time n.

Q2: By the answer to Q1, all values in the Markov Chain are not independent of each other because $P(X_{n+1} | X_n) \ne P(X_{n+1})$. After enough iterations, the chain (usually) converges in distribution so they would be identically distributed.

That's all I know.

$\endgroup$
  • $\begingroup$ I don't think your answer to Q3 is correct, because it's not hard to construct Markov chains ($X_i$) in which, say, $X_4$ and $X_2$ are perfectly correlated. Am I misunderstanding your assertion? $\endgroup$ – whuber Nov 4 '15 at 18:55
  • $\begingroup$ Thank you for your reply, but could you provide a link to some resource where the rational for answer 3 is presented formally. $\endgroup$ – Srishti M Nov 5 '15 at 0:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.