I am trying to understand how this Markov chain would arise.

denote $p_{ij}^{(n)}$ the probability of going from position $i$ to $j$ in $n$ steps.
Assume $p_{ij}^{(n)}>0$ and $p_{ji}^{(n)} > 0$ is it possible for state $i$ to be transient?

I do not quite know how to think of this, here is my current take:
Maybe given the properties of $p_{ij}^{(n)}>1/5$ and $p_{ji}^{(n)} > 1/5\space \forall n\geq 0$, this implies that $i$ will always rotate between positions $i$ and $j$ given the proba is not 0. However I am also thinking that there is some proba that we might go from state $i$ to $k$ from which $k$ is an element of a closed subset when $i$ is not, however would that mean that $\lim_{n \rightarrow \infty} p_{ij}^{(n)}=0$ and $\lim_{n \rightarrow \infty}p_{ji}^{(n)} = 0$


Let us assume that the Markov Chain is finite state, time-homogeneous.

If $p_{ij}^{(m)}>0$, and $p_{ji}^{(n)}>0$ for some integers $m\geq 1$ and $n\geq 1$, then we say that states $i$ and $j$ communicate with each other.

Case-1: Markov chain is irreducible: All the states belong to a single closed communicating class. States belonging to a closed communicating class are recurrent. Hence, both the states $i$ and $j$ are positive recurrent.

Case-2: Markov chain is not irreducible: The state space is reducible into closed and non-closed communicating classes.

For a finite state, time-homogeneous Markov chains, there must be at least one closed communicating class.

If states $i$ and $j$ belong to a closed communicating class, then they are positive recurrent.

The states $i$ and $j$ are transient, if both of them belong to a non-closed communicating class.

  • $\begingroup$ So in your last sentence, i and j being transient, if both of them in a non-closed communicating cass, can they still have the $p_{ij}^{(m)} > 0$ and $p_{ji}^{(n)} > 0$ for all positive n? $\endgroup$ Feb 2 '17 at 18:07
  • 1
    $\begingroup$ for all positive n? - no. only for some n. Suppose that the state space is reducible into one two communicating classes: one closed and the other non-closed. If the process is currently moving among the states of a non-closed communicating class, after some time, it eventually escapes from that class to the closed communicating class. When once it happens, the process moves over the states of closed communicating class. In the long run, we find the process moving among the recurrent states only. $\endgroup$
    – L.V.Rao
    Feb 3 '17 at 2:39
  • 1
    $\begingroup$ I spent some time meditating on this and understand how this works! your awnsers were very useful! $\endgroup$ Feb 3 '17 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.