1
$\begingroup$

Does there exist an irreducible discrete time Markov chain such that all limiting probabilities exist, i.e. $\lim\limits_{k \to \infty} \mathbb{P}(X_k = i)$ exists for all states $i$, and $\sum_{i} \lim\limits_{k \to \infty} \mathbb{P}(X_k = i) = \frac{1}{2}$? I'm currently stuck here and not sure how to prove/disprove this. Is there any suggestion on how I should proceed?

$\endgroup$
1
  • $\begingroup$ If I understand your notation and terminology, it seems to me the sum of limits has to b $1.$ Terminology can differ among texts. If you allow periodic chains maybe 1/2 is possible. $\endgroup$
    – BruceET
    Commented Mar 26, 2021 at 7:08

1 Answer 1

2
$\begingroup$

Consider the $2\times 2$ transition matrix, shown below. In R, matrix multiplication is denoted %*% We show the transition matrix of an irreducible ergodic chain, and how to obtain its limiting distribution.

First approach: Take higher powers of transition matrix, until all rows of $\mathbf{P}^n$ become (nearly) the same.

P = matrix(c(.4, .6,
             .7, .3), byrow=T, nrow=2)
P.2 = P %*% P;  P.4 = P.2 %*% P.2;  P.4
       [,1]   [,2]
[1,] 0.5422 0.4578
[2,] 0.5341 0.4659
P.8 = P.4 %*% P.4;  P.16 = P.8 %*% P.8;  P.16
          [,1]      [,2]
[1,] 0.5384615 0.4615385
[2,] 0.5384615 0.4615385

So the 16th power of the transition matrix shows that the steady state, thus limiting, distribution is the vector $\sigma = (0.5384615,\, 0.4615385).$

Another approach, using eigenvectors: The left eigenvector with the smallest modulus, listed first in R output, is proportional to $\sigma.$ Because R finds right eigenvectors, we use the transpose of the transition matrix $\mathbf{P}.$

g = eigen(t(P))$vector[,1]; g  # first column of matris of eigenvectors
[1] 0.7592566 0.6507914
sg = g/sum(g);  sg
[1] 0.5384615 0.4615385

sg %*% P                       # verification
          [,1]      [,2]
[1,] 0.5384615 0.4615385

Third approach is possible tor a 2-state irreducible ergodic Markov Chain. Starting in state 1, the mean waiting time to transition to state 2 is (by geometric distribution $1/.6.$ Then the mean waiting time to return to state 1 is $1/.7.$ Thus on average, over the long run, the chain spends $\frac{1/.6}{1/.6 + 1/.7} = \frac{7}{13} = 0.5384615$ of its time in state 1.

1/6/(1/6 + 1/7)
[1] 0.5384615
$\endgroup$
2
  • $\begingroup$ I'm not sure whether I understand the reasoning above. The sum of the limiting probabilities in all the examples is still 1. So I guess the question above is not true? Do you have any suggestion how to disprove it? $\endgroup$
    – S10000
    Commented Mar 26, 2021 at 14:05
  • $\begingroup$ Not sure I understand question. My answer is is correct and contains info you might find useful, but maybe irrelevant to current question. $\endgroup$
    – BruceET
    Commented Mar 26, 2021 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.