2
$\begingroup$

Suppose we have a Markov chain $X$, with transition probabilities $T_{ij} = P(X_{n+1}=i\mid X_{n}=j )$.

Let $(Y_{1}, Y_{2}, .., Y_{N})$ represent the reversed Markov chain of $X$, such that $Y_{n} = X_{N-n}$. Then, $$P(Y_{n+1}=j \mid Y_{n}=i) = T_{ij}\frac{P(Y_{n+1}=j)}{P(Y_{n}=i)}.$$ I understand that this result holds if X is homogeneous and is irrelevant whether X is irreducible.

If $X$ is an irreducible Markov chain then $T_{ij}^{(n)} \geq 0$ for each possible pair $i, j$ such that $i \neq j$, for some integer $n$.

Should one use the idea of detailed balance to show the result that an irreducible (but not necessarily homogeneous) chain must be reversible? I am unsure if the above homogeneous case helps any explanation in the context of the irreducible case.

Thank you in advance for your help and thoughts.

$\endgroup$
2
  • 2
    $\begingroup$ (Note that Markov was a person, so it is proper to capitalise his name when referring to a Markov chain. Same with Bayes' theorem, Fisher's test, etc.) $\endgroup$
    – Ben
    Apr 12, 2020 at 1:54
  • 4
    $\begingroup$ An irreducable Markov chain is not always reversible. E.g., the Markov chain corresponding to a top-to-random shuffle on n cards (at each step taking the top card and inserting it at one of the n positions in the deck chosen uniformly at random) is irreducible, aperiodic but not reversible. $\endgroup$
    – statmerkur
    Apr 12, 2020 at 9:34

1 Answer 1

3
$\begingroup$

Simplest example of a non-reversible irreducible Markov chain is a deterministic one with three states, that goes $1,2,3,1,2,3,1,2,3,...$

Observing that chain, you can tell if time is going forward or backward.

I think your definition of an irreducible Markov chain is flawed, also. In fact, it's vacuous; $T_{ij} \geq 0$ no matter what. You might be thinking of aperiodicity, one definition of which is that for any fixed $i,j$, $T_{ij}^{(n)}$ is always $>0$ for large $n$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.