Infinite Markov Chain and finding the Stationary Distribution?

Question

I have the following transition matrix for my Markov Chain:

$$ P= \begin{pmatrix} 1/2 & 1/2 & 0&0&0& \cdots \\ 2/3 & 0 & 1/3&0&0&\cdots \\ 3/4 & 0& 0&1/4&0&\cdots \\ 4/5 & 0&0&0&1/5&\cdots\\ 5/6&0&0&0&0&\cdots\\ \vdots&\vdots&\vdots&\vdots&\vdots&\ddots \end{pmatrix} $$

defined by

$$ P_{ij} = \begin{cases} \frac{i}{i+1}, & \text{if j=1} \\ \frac{1}{i+1}, & \text{j=i+1}\\ 0, & \text{otherwise.} \end{cases} $$

I've tried finding a pattern for just a 5x5 version of this but I can't seem to find anything unique. In fact, all of these zeroes seem to take on values at higher powers of the matrix.

The only pattern I see is that each element in the 1st column becomes the sum of 1/2 the element in the column plus 1/i+1. They seem to approach .582 for all elements... But how would I show this in the infinite case? I can't envision how I would multiply by the stationary distribution $\pi$.

Can anyone give me some insight on where I should go from here? Thank you greatly in advance!

EDIT: Forgot the actual question. Ugh I'm tired :/

(a) Does the chain have a stationary distribution? If yes, exhibit the distribution. If no, explain why.

(b) Classify the states of the chain.

Answer 1

There is a technique to reduce the number of equations to solve by one. Note that if $x$ satisfies $xP=x$ then (c$x$)$P$ = c$x$, $\forall$c.

Let c = $\frac{1}{\sum_j x_j}$

Then we can have $x$ = $(1, x_1, x_2, ...)$

and $\pi = \frac{1}{1+x_2+x_3+...+x_4} (1,x_2,x_3,...,x_k)$.

---

Actually solving:

We know the pattern of P which is given in my question. When you multiply,

$\pi P = \pi$

You get the system of equations;

$\frac{1}{2}\pi_1 + \frac{2}{3}\pi_2 + \frac{3}{4}\pi_3+... = \pi_1$

$\frac{1}{2} \pi_1 = \pi_2$

$\frac{1}{3}\pi_2 = \pi_3$

$\frac{1}{4}\pi_3 = \pi_4$

Continue in this manner indefinitely. The pattern is clear, though.

Now, as explained above, set $\pi_1 = 1$.

Then we get $\pi_1 = 1$, $\pi_2 = \frac{1}{2}$, $\pi_3=\frac{1}{6}$, $\pi_4 = \frac{1}{24}$,$...$

Which you can express as factorials.

$\pi_1 = \frac{1}{1!}$, $\pi_2 = \frac{1}{2!}$, $\pi_3=\frac{1}{3!}$, $\pi_4 = \frac{1}{4!}$,$...$

This is actually a Maclaurin Series!

$\sum_{j=0}^{\infty} \frac{1}{j!} = e$

and in our case we start with $j=1$ so we have,

$\sum_{j=1}^{\infty} \frac{1}{j!} = e-1$

So we have,

$\pi = \frac{1}{e-1} (1, \frac{1}{2},\frac{1}{6},...,\frac{1}{j!})$