0
$\begingroup$

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following:

Because $\pi_i$ is the long-run proportion of time that the chain is in state $i$, and $P^n_{i, j}$ is the long-run proportion of time when the Markov chain is in state $i$ that it will be in state $j$ after $n$ transition

$$\pi_i P^n_{i, j} = \text{long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions}$$

enter image description here

But it seems to me that this is describing $\pi_i$ and $P^n_{i, j}$ identically? So what is the difference supposed to be?

I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$

2 Answers 2

1
$\begingroup$

If you have $K$ states there are $K$ numbers $\pi_i$ and (for each $n$) $K^2$ numbers $P_{i,j}^n$. The $\pi_i$ are long-term probabilities of being in state $i$ and the $P_{i,j}$ are long-term conditional probabilities of moving from state $j$ to state $i$

There's a close relationship between the sets of numbers: everything is the way it is because it got that way. If there's a stationary distribution and you pick $j$ from that stationary distribution, then $\pi_i=\sum_j P^n_{i,j}\pi_j$ for any $n$ because that's what it means to be a stationary distribution -- if you're in it, you stay in it.

If the chain has a unique stationary distribution and converges to it, then as $n\to\infty$, for any $j$, $P^n_{i,j}\to\pi_i$. Again, that's what it means: that you can start chains anywhere, and you they will eventually end up with close to the stationary distribution. That's what the theorem is saying. But there are assumptions; it's not just true by definition and it's not just two ways of saying the same thing

To see that the assumptions matter, suppose you had a chain with two subsets of states ('red' and 'blue') and it always moved to a state of a different colour at every transition. There would be no vector $\pi_i$ such that $P^n_{i,j}\to\pi_i$ for every $j$, because $P^n_{i,j}$ would always be zero for different-coloured states $(i,j)$ and even $n$, or for same-coloured states $(i,j)$ and odd $n$. That's why the theorem requires that $n$ is chosen to give $P^n_{i,j}>0$, and positive recurrence plus connectedness guarantees this is possible

Conversely, imagine a chain that always transitions to a state of the same colour. In that case, knowing that a red state $i$ is positive recurrent doesn't tell you that a blue state $j$ is positive recurrent because you can't get there from here. Again, the conditions of the theorem are needed for the result.

$\endgroup$
0
$\begingroup$

It is stated in the first sentence, but I agree that it is a bit confusing.

As I remember it, $\pi_{i}$ is the proportion of time that the Markov chain is in state $i$. For example, if $\pi_{i} = 0.5$ then the Markov chain is in state $i$ half of the time. In other words, if you look at the chain at random moments in time, you expect the chain to be in state $i$ in 50 percent of those instances.

$P^{n}$ is the $n$-th power of the transition matrix, which gives the probabilities after $n$ transitions. So $P_{i,j}^{n}$ is the probability that you start in state $i$ and after $n$ transitions are in state $j$.

$\endgroup$
2
  • $\begingroup$ Isn’t $\pi_i$ a vector? Or am I misunderstanding this? $\endgroup$ May 5, 2020 at 14:46
  • $\begingroup$ $\pi = (\pi_{i})_{i=1}^{N}$ where $N$ is the number of states is a vector containing all the long-run proportions of time that the chain is in a certain state. The proportion for state $i$ is denoted by $\pi_{i}$ and they sum up to 1, i.e. $\sum_{i=1}^{N} \pi_{i} = 1$. $\endgroup$
    – user585522
    May 5, 2020 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.