# Question regarding limiting probability

Does there exist an irreducible discrete time Markov chain such that all limiting probabilities exist, i.e. $$\lim\limits_{k \to \infty} \mathbb{P}(X_k = i)$$ exists for all states $$i$$, and $$\sum_{i} \lim\limits_{k \to \infty} \mathbb{P}(X_k = i) = \frac{1}{2}$$? I'm currently stuck here and not sure how to prove/disprove this. Is there any suggestion on how I should proceed?

• If I understand your notation and terminology, it seems to me the sum of limits has to b $1.$ Terminology can differ among texts. If you allow periodic chains maybe 1/2 is possible. Commented Mar 26, 2021 at 7:08

Consider the $$2\times 2$$ transition matrix, shown below. In R, matrix multiplication is denoted %*% We show the transition matrix of an irreducible ergodic chain, and how to obtain its limiting distribution.

First approach: Take higher powers of transition matrix, until all rows of $$\mathbf{P}^n$$ become (nearly) the same.

P = matrix(c(.4, .6,
.7, .3), byrow=T, nrow=2)
P.2 = P %*% P;  P.4 = P.2 %*% P.2;  P.4
[,1]   [,2]
[1,] 0.5422 0.4578
[2,] 0.5341 0.4659
P.8 = P.4 %*% P.4;  P.16 = P.8 %*% P.8;  P.16
[,1]      [,2]
[1,] 0.5384615 0.4615385
[2,] 0.5384615 0.4615385


So the 16th power of the transition matrix shows that the steady state, thus limiting, distribution is the vector $$\sigma = (0.5384615,\, 0.4615385).$$

Another approach, using eigenvectors: The left eigenvector with the smallest modulus, listed first in R output, is proportional to $$\sigma.$$ Because R finds right eigenvectors, we use the transpose of the transition matrix $$\mathbf{P}.$$

g = eigen(t(P))\$vector[,1]; g  # first column of matris of eigenvectors
[1] 0.7592566 0.6507914
sg = g/sum(g);  sg
[1] 0.5384615 0.4615385

sg %*% P                       # verification
[,1]      [,2]
[1,] 0.5384615 0.4615385


Third approach is possible tor a 2-state irreducible ergodic Markov Chain. Starting in state 1, the mean waiting time to transition to state 2 is (by geometric distribution $$1/.6.$$ Then the mean waiting time to return to state 1 is $$1/.7.$$ Thus on average, over the long run, the chain spends $$\frac{1/.6}{1/.6 + 1/.7} = \frac{7}{13} = 0.5384615$$ of its time in state 1.

1/6/(1/6 + 1/7)
[1] 0.5384615

• I'm not sure whether I understand the reasoning above. The sum of the limiting probabilities in all the examples is still 1. So I guess the question above is not true? Do you have any suggestion how to disprove it? Commented Mar 26, 2021 at 14:05
• Not sure I understand question. My answer is is correct and contains info you might find useful, but maybe irrelevant to current question. Commented Mar 26, 2021 at 16:05