limit and stationary distribution of a Markov chain - Cross Validated most recent 30 from stats.stackexchange.com 2019-08-18T05:05:23Z https://stats.stackexchange.com/feeds/question/404509 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/404509 0 limit and stationary distribution of a Markov chain Jin Yu Li https://stats.stackexchange.com/users/182447 2019-04-23T01:11:04Z 2019-04-24T05:08:08Z <p>Consider a Markov chain on the non-negative integers with transition probabilities 􏰀<span class="math-container">$1/2$</span> if <span class="math-container">$y=x+1$</span> and <span class="math-container">$1/2$</span> if <span class="math-container">$y=0$</span>. Find <span class="math-container">$\lim_{n \to \infty} P(X_{n}=0)$</span>.</p> <p>Is this limit the same as the stationary distribution? What's the relationship between the two?</p> https://stats.stackexchange.com/questions/404509/-/404739#404739 1 Answer by Sanket Agrawal for limit and stationary distribution of a Markov chain Sanket Agrawal https://stats.stackexchange.com/users/166295 2019-04-24T05:08:08Z 2019-04-24T05:08:08Z <p>Given is the Markov Chain <span class="math-container">$\{X_n, n\ge1\}$</span>, with its T.P.M described as,</p> <p><span class="math-container">$p_{ij}^{(1)} = P(X_n = j \mid X_{n-1} = i) = \begin{cases} \frac{1}{2} &amp; \text{if$j = i + 1$}\\ \frac{1}{2} &amp; \text{if$j = 0$}\\ 0 &amp; \text{otherwise} \end{cases}$</span></p> <p>We need to obtain the limiting distribution of the Markov Chain, i.e. <span class="math-container">$\displaystyle \lim_{n\to\infty} p_{ij}^{(n)}$</span> where <span class="math-container">$p_{ij}^{(n)}$</span> is the n-step transition probability.</p> <p>Consider this,</p> <p><span class="math-container">$p_{ij}^{(n)} = P(X_n = j \mid X_0 = i) = \displaystyle \sum_{k = 0}^{\infty}P(X_n = j \mid X_{n-1} = k).P(X_{n-1} = k \mid X_0 = i) ; j \ne 0$</span></p> <p><span class="math-container">$p_{ij}^{(n)} = \displaystyle \sum_{k = 0}^{\infty}p_{kj}^{(1)}.p_{ik}^{(n-1)} ; j \ne 0$</span></p> <p>But <span class="math-container">$p_{kj}^{(1)} = \frac{1}{2}$</span> if <span class="math-container">$k = j - 1$</span> and <span class="math-container">$0$</span> otherwise, so we get the following recurrence relation, </p> <p><span class="math-container">$p_{ij}^{(n)} = \dfrac{1}{2}.p_{ij-1}^{(n-1)} ; j \ne 0$</span>. </p> <p>Denote by <span class="math-container">$p_{ij}$</span> the limiting probability <span class="math-container">$\displaystyle \lim_{n\to\infty}p_{ij}^{(n)}$</span>. Taking the limits of the above equation, we get</p> <p><span class="math-container">\begin{align} \displaystyle \lim_{n\to\infty}p_{ij}^{(n)} = \displaystyle \lim_{n\to\infty} \dfrac{1}{2}.p_{ij-1}^{(n-1)} \\ p_{ij} = \dfrac{1}{2}.p_{ij-1} = \left( \frac{1}{2}\right)^jp_{i0} \tag 1\\ \end{align}</span></p> <p><span class="math-container">$\text{ where } p_{i0} = \displaystyle \lim_{n\to\infty}P(X_n = 0 \mid X_0 = i) \text{ and } j \ne 0$</span></p> <p>Again, <span class="math-container">$P(X_n = 0 \mid X_0 = i) = \displaystyle \sum_{k = 0}^{\infty}P(X_n = 0 \mid X_{n-1} = k).P(X_{n-1} = k \mid X_0 = i)$</span></p> <p><span class="math-container">$p_{i0}^{(n)} = \displaystyle \sum_{k = 0}^{\infty}p_{k0}^{(1)}.p_{ik}^{(n-1)}$</span></p> <p>But <span class="math-container">$p_{k0}^{(1)} = \frac{1}{2} \text{ for all } k \text{ and } \displaystyle \sum_{k = 0}^{\infty}p_{ik}^{(n-1)} = 1$</span>, we get <span class="math-container">$p_{i0}^{(n)} = \frac{1}{2}$</span>. And thus,</p> <p><span class="math-container">$p_{i0} = \displaystyle \lim_{n\to\infty}p_{i0}^{(n)} = \dfrac{1}{2}$</span>. Now putting this value in <span class="math-container">$(1)$</span>, we get,</p> <p><span class="math-container">$p_{ij} = \left(\dfrac{1}{2}\right)^{j+1}$</span>.</p> <p>From this, we notice that the limiting probability <span class="math-container">$p_{ij}$</span> is independent of the initial state <span class="math-container">$i$</span>. And thus all the rows in the corresponding TPM will be identical, making it also the stationary probabilities.</p>