$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following:

Because $$\pi_i$$ is the long-run proportion of time that the chain is in state $$i$$, and $$P^n_{i, j}$$ is the long-run proportion of time when the Markov chain is in state $$i$$ that it will be in state $$j$$ after $$n$$ transition

$$\pi_i P^n_{i, j} = \text{long-run proportion of time the chain is in i and will be in j after n transitions}$$

But it seems to me that this is describing $$\pi_i$$ and $$P^n_{i, j}$$ identically? So what is the difference supposed to be?

I would greatly appreciate it if people would please take the time to clarify this.

If you have $$K$$ states there are $$K$$ numbers $$\pi_i$$ and (for each $$n$$) $$K^2$$ numbers $$P_{i,j}^n$$. The $$\pi_i$$ are long-term probabilities of being in state $$i$$ and the $$P_{i,j}$$ are long-term conditional probabilities of moving from state $$j$$ to state $$i$$

There's a close relationship between the sets of numbers: everything is the way it is because it got that way. If there's a stationary distribution and you pick $$j$$ from that stationary distribution, then $$\pi_i=\sum_j P^n_{i,j}\pi_j$$ for any $$n$$ because that's what it means to be a stationary distribution -- if you're in it, you stay in it.

If the chain has a unique stationary distribution and converges to it, then as $$n\to\infty$$, for any $$j$$, $$P^n_{i,j}\to\pi_i$$. Again, that's what it means: that you can start chains anywhere, and you they will eventually end up with close to the stationary distribution. That's what the theorem is saying. But there are assumptions; it's not just true by definition and it's not just two ways of saying the same thing

To see that the assumptions matter, suppose you had a chain with two subsets of states ('red' and 'blue') and it always moved to a state of a different colour at every transition. There would be no vector $$\pi_i$$ such that $$P^n_{i,j}\to\pi_i$$ for every $$j$$, because $$P^n_{i,j}$$ would always be zero for different-coloured states $$(i,j)$$ and even $$n$$, or for same-coloured states $$(i,j)$$ and odd $$n$$. That's why the theorem requires that $$n$$ is chosen to give $$P^n_{i,j}>0$$, and positive recurrence plus connectedness guarantees this is possible

Conversely, imagine a chain that always transitions to a state of the same colour. In that case, knowing that a red state $$i$$ is positive recurrent doesn't tell you that a blue state $$j$$ is positive recurrent because you can't get there from here. Again, the conditions of the theorem are needed for the result.

It is stated in the first sentence, but I agree that it is a bit confusing.

As I remember it, $$\pi_{i}$$ is the proportion of time that the Markov chain is in state $$i$$. For example, if $$\pi_{i} = 0.5$$ then the Markov chain is in state $$i$$ half of the time. In other words, if you look at the chain at random moments in time, you expect the chain to be in state $$i$$ in 50 percent of those instances.

$$P^{n}$$ is the $$n$$-th power of the transition matrix, which gives the probabilities after $$n$$ transitions. So $$P_{i,j}^{n}$$ is the probability that you start in state $$i$$ and after $$n$$ transitions are in state $$j$$.

• Isn’t $\pi_i$ a vector? Or am I misunderstanding this? May 5, 2020 at 14:46
• $\pi = (\pi_{i})_{i=1}^{N}$ where $N$ is the number of states is a vector containing all the long-run proportions of time that the chain is in a certain state. The proportion for state $i$ is denoted by $\pi_{i}$ and they sum up to 1, i.e. $\sum_{i=1}^{N} \pi_{i} = 1$. May 5, 2020 at 17:29