# Determine if the following Markov chain is positive recurrent, null recurrent or transcient

We consider the Markov Chain with transition probabilities $$p(i,0)=\frac{1}{i^2 +2},\qquad p(i,i+1)= \frac{i^2 +1}{i^2 +2}.$$

Determine if this Markov chain is positive recurrent, null recurrent or transcient.

My attempt: Since all states are connected to $$0$$, then it is sufficient to determine if $$0$$ is a positive recurring state. Consider $$T_{0}$$ the hitting time, that is $$T_{0}=\inf\left\{m\geq 1\: :\: X_{m}=0\right\}.$$ Note that $$\mathbb{P}(T_{0}=n|X_{0}=0)=\left(\frac{1}{2}\times\frac{2}{3}\times\cdots\times\frac{(n-2)^2+1}{(n-2)^{2}+2}\right)\left(\frac{1}{(n-1)^{2}+2}\right)$$ Therefore, we have $$\mathbb{E}(T_{0}|X_{0}=0)=\sum_{n=1}^{\infty}n\times \left(\frac{1}{2}\times\frac{2}{3}\times\cdots\times\frac{(n-2)^2+1}{(n-2)^{2}+2}\right)\left(\frac{1}{(n-1)^{2}+2}\right).$$ I need to determine if this series converges or diverges. I have tried to limit it superiorly and inferiorly but I have not found good bounds.

Handle the product in the summation by taking logs:

\eqalign{ \log \prod_{i=0}^{n-2} \frac{i^2+1}{i^2+2} &= \sum_{i=0}^n \log\left(1 - \frac{1}{i^2+2}\right)\\ &\ge -\sum_{i=0}^n \frac{1}{i^2+2} \\ &\gt -\sum_{i=0}^\infty \frac{1}{i^2+2} \\ &\gt -\frac{1}{2} - \sum_{i=1}^\infty \frac{1}{i^2} \\ &= -\frac{3+\pi^2}{6}. }

Consequently you can underestimate the sum as

\eqalign { \sum_{n=1}^\infty n \prod_{i=0}^{n-2} \frac{i^2+1}{i^2+2} \frac{1}{(n-1)^2+2} & \gt \frac{1}{e^{(3+\pi^2)/6}} \sum_{n=1}^\infty \frac{n}{(n-1)^2+2}. }

The right hand side diverges (compare it to $$\int_1^\infty \frac{x}{x^2+2}\mathrm{d}x$$).

• I'd have been inclined to try to cancel all the common terms in numerator and denominator first. (denominator of 1/2 cancels with numerator of 2/3, etc). Dec 4, 2019 at 11:56
• @Glen_b That's exactly what I thought, too, until I looked at the subsequent terms--there is no telescoping.
– whuber
Dec 4, 2019 at 15:08
• You showed that if $\left\{X_{t}\right\}$ is recurrent, then it is null recurrent. But you do not showed that $\left\{X_{t}\right\}$ is recurrent. This chain could be transient. Dec 9, 2019 at 14:49
• Diego, I didn't show anything about recurrency: I have only responded to your specific question, "I need to determine if this series converges or diverges."
– whuber
Dec 9, 2019 at 14:52