Let $X$ be a Markov chain with a state space $S={\{0,1,2,... \}}$ and a transition matrix $P$ with given $p_{i,0}=\frac{i}{i+1}$ and $p_{i,i+1}=\frac{1}{i+1}$, for $i=0,1,2,...$. Find out which states are transient, null and not-null. Find stationary distribution.

I have that $p_{0,0}=0, p_{1,0}=\frac{1}{2}, p_{2,0}=\frac{2}{3}, p_{3,0}=\frac{3}{4}, ...$ and $p_{0,1}=1, p_{1,2}=\frac{1}{2}, p_{2,3}=\frac{1}{3}, p_{3,4}=\frac{1}{4},...$.

All the states are persistent so no states are transient

A state is null $\iff \lim_{n \to \infty} p_{i,i}(n)=0$, so state $0$ is null (because $\lim_{n \to \infty} \frac{1}{n+1}=0$).

From now I don't know how to determine whether states $1,...,J$ are null or not-null. My idea was to let $T_i$ be the time of first return to $i$, but I'm not sure how to continue. Please help me with the above exercise and send any tips.

  • $\begingroup$ Please add the self-study tag since this is related with an exercise. $\endgroup$ – Xi'an Jan 12 at 14:25

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.