We consider the Markov Chain with transition probabilities $$ p(i,0)=\frac{1}{i^2 +2},\qquad p(i,i+1)= \frac{i^2 +1}{i^2 +2}. $$

Determine if this Markov chain is positive recurrent, null recurrent or transcient.

enter image description here

My attempt: Since all states are connected to $0$, then it is sufficient to determine if $0$ is a positive recurring state. Consider $T_{0}$ the hitting time, that is $$T_{0}=\inf\left\{m\geq 1\: :\: X_{m}=0\right\}.$$ Note that $$ \mathbb{P}(T_{0}=n|X_{0}=0)=\left(\frac{1}{2}\times\frac{2}{3}\times\cdots\times\frac{(n-2)^2+1}{(n-2)^{2}+2}\right)\left(\frac{1}{(n-1)^{2}+2}\right) $$ Therefore, we have $$ \mathbb{E}(T_{0}|X_{0}=0)=\sum_{n=1}^{\infty}n\times \left(\frac{1}{2}\times\frac{2}{3}\times\cdots\times\frac{(n-2)^2+1}{(n-2)^{2}+2}\right)\left(\frac{1}{(n-1)^{2}+2}\right). $$ I need to determine if this series converges or diverges. I have tried to limit it superiorly and inferiorly but I have not found good bounds.


1 Answer 1


Handle the product in the summation by taking logs:

$$\eqalign{ \log \prod_{i=0}^{n-2} \frac{i^2+1}{i^2+2} &= \sum_{i=0}^n \log\left(1 - \frac{1}{i^2+2}\right)\\ &\ge -\sum_{i=0}^n \frac{1}{i^2+2} \\ &\gt -\sum_{i=0}^\infty \frac{1}{i^2+2} \\ &\gt -\frac{1}{2} - \sum_{i=1}^\infty \frac{1}{i^2} \\ &= -\frac{3+\pi^2}{6}. }$$

Consequently you can underestimate the sum as

$$\eqalign { \sum_{n=1}^\infty n \prod_{i=0}^{n-2} \frac{i^2+1}{i^2+2} \frac{1}{(n-1)^2+2} & \gt \frac{1}{e^{(3+\pi^2)/6}} \sum_{n=1}^\infty \frac{n}{(n-1)^2+2}. }$$

The right hand side diverges (compare it to $\int_1^\infty \frac{x}{x^2+2}\mathrm{d}x$).

  • $\begingroup$ I'd have been inclined to try to cancel all the common terms in numerator and denominator first. (denominator of 1/2 cancels with numerator of 2/3, etc). $\endgroup$
    – Glen_b
    Dec 4, 2019 at 11:56
  • 1
    $\begingroup$ @Glen_b That's exactly what I thought, too, until I looked at the subsequent terms--there is no telescoping. $\endgroup$
    – whuber
    Dec 4, 2019 at 15:08
  • $\begingroup$ You showed that if $\left\{X_{t}\right\}$ is recurrent, then it is null recurrent. But you do not showed that $\left\{X_{t}\right\}$ is recurrent. This chain could be transient. $\endgroup$ Dec 9, 2019 at 14:49
  • $\begingroup$ Diego, I didn't show anything about recurrency: I have only responded to your specific question, "I need to determine if this series converges or diverges." $\endgroup$
    – whuber
    Dec 9, 2019 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.