Let $X$ be a Markov chain with a state space $S={\{0,1,2,... \}}$ and a transition matrix $P$ with given $p_{i,0}=\frac{i}{i+1}$ and $p_{i,i+1}=\frac{1}{i+1}$, for $i=0,1,2,...$. Find out which states are transient, null and not-null. Find stationary distribution.

I have that $p_{0,0}=0, p_{1,0}=\frac{1}{2}, p_{2,0}=\frac{2}{3}, p_{3,0}=\frac{3}{4}, ...$ and $p_{0,1}=1, p_{1,2}=\frac{1}{2}, p_{2,3}=\frac{1}{3}, p_{3,4}=\frac{1}{4},...$.

All the states are persistent so no states are transient

A state is null $\iff \lim_{n \to \infty} p_{i,i}(n)=0$, so state $0$ is null (because $\lim_{n \to \infty} \frac{1}{n+1}=0$).

From now I don't know how to determine whether states $1,...,J$ are null or not-null. My idea was to let $T_i$ be the time of first return to $i$, but I'm not sure how to continue.

  • $\begingroup$ Please add the self-study tag since this is related with an exercise. $\endgroup$ – Xi'an Jan 12 '19 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.