2
$\begingroup$

I have the following transition matrix for my Markov Chain:

$$ P= \begin{pmatrix} 1/2 & 1/2 & 0&0&0& \cdots \\ 2/3 & 0 & 1/3&0&0&\cdots \\ 3/4 & 0& 0&1/4&0&\cdots \\ 4/5 & 0&0&0&1/5&\cdots\\ 5/6&0&0&0&0&\cdots\\ \vdots&\vdots&\vdots&\vdots&\vdots&\ddots \end{pmatrix} $$

defined by

$$ P_{ij} = \begin{cases} \frac{i}{i+1}, & \text{if j=1} \\ \frac{1}{i+1}, & \text{j=i+1}\\ 0, & \text{otherwise.} \end{cases} $$

I've tried finding a pattern for just a 5x5 version of this but I can't seem to find anything unique. In fact, all of these zeroes seem to take on values at higher powers of the matrix.

The only pattern I see is that each element in the 1st column becomes the sum of 1/2 the element in the column plus 1/i+1. They seem to approach .582 for all elements... But how would I show this in the infinite case? I can't envision how I would multiply by the stationary distribution $\pi$.

Can anyone give me some insight on where I should go from here? Thank you greatly in advance!

EDIT: Forgot the actual question. Ugh I'm tired :/

(a) Does the chain have a stationary distribution? If yes, exhibit the distribution. If no, explain why.

(b) Classify the states of the chain.

$\endgroup$
  • 2
    $\begingroup$ Hint: $$0.581977\ldots = \frac{1}{e-1} = \frac{1}{1+1/2+1/6+1/24+\cdots}.$$ $\endgroup$ – whuber Mar 1 '18 at 2:34
  • 1
    $\begingroup$ I got the answer! Thanks for your help. I'll post my own solution to this question for anyone having trouble with this in a bit! Thanks whuber, you've been super helpful in my studies :) $\endgroup$ – Nicklovn Mar 1 '18 at 16:02
1
$\begingroup$

There is a technique to reduce the number of equations to solve by one. Note that if $x$ satisfies $xP=x$ then (c$x$)$P$ = c$x$, $\forall$c.

Let c = $\frac{1}{\sum_j x_j}$

Then we can have $x$ = $(1, x_1, x_2, ...)$

and $\pi = \frac{1}{1+x_2+x_3+...+x_4} (1,x_2,x_3,...,x_k)$.


Actually solving:

We know the pattern of P which is given in my question. When you multiply,

$\pi P = \pi$

You get the system of equations;

$\frac{1}{2}\pi_1 + \frac{2}{3}\pi_2 + \frac{3}{4}\pi_3+... = \pi_1$

$\frac{1}{2} \pi_1 = \pi_2$

$\frac{1}{3}\pi_2 = \pi_3$

$\frac{1}{4}\pi_3 = \pi_4$

Continue in this manner indefinitely. The pattern is clear, though.

Now, as explained above, set $\pi_1 = 1$.

Then we get $\pi_1 = 1$, $\pi_2 = \frac{1}{2}$, $\pi_3=\frac{1}{6}$, $\pi_4 = \frac{1}{24}$,$...$

Which you can express as factorials.

$\pi_1 = \frac{1}{1!}$, $\pi_2 = \frac{1}{2!}$, $\pi_3=\frac{1}{3!}$, $\pi_4 = \frac{1}{4!}$,$...$

This is actually a Maclaurin Series!

$\sum_{j=0}^{\infty} \frac{1}{j!} = e$

and in our case we start with $j=1$ so we have,

$\sum_{j=1}^{\infty} \frac{1}{j!} = e-1$

So we have,

$\pi = \frac{1}{e-1} (1, \frac{1}{2},\frac{1}{6},...,\frac{1}{j!})$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.