1
$\begingroup$

The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with

$p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is

P =$\begin{pmatrix} \frac{1}{4} & \frac{3}{4} & 0 & 0 & ...\\\ \frac{1}{4} & 0 & \frac{3}{4} & 0 & ... \\\ \frac{1}{4} & 0 & 0 & \frac{3}{4} &... \\\ \vdots & \vdots&\vdots&\vdots& \ddots \end{pmatrix}$

Find the irreducible classes of intercommunicating states. For each class, state:

(a) whether it is transient, positive recurrent or null recurrent (hint - think about the distribution of the return times - say to state 0 - in this case. From there, you can work out whether the states have a finite or infinite expected return time. Can you work out what sort of states you have here?)

(b) its periodicity.

I have tried to approach the first part with a state space diagram and have found that only state 0 and 1 intercommunicate and that the rest of the states do not. But, it is possible to return to every state at some point (say we got to 3, we can then go to state 4, then state 0, 1, 2 and end up at 3 again. So would I put {0,1,2,3,4, ...} into one class?

I guess I am also not 100% sure about how to classify the states.

$\endgroup$
2
$\begingroup$

All states communicate by the Chapman-Kolmogorov equation. For any $i,k$

\begin{align*} P(X_n = k \mid X_1 = i) &= \sum_j P(X_n = k \mid X_{2} = j) P(X_2 = j \mid X_1 = i) \\ &\ge P(X_n = k \mid X_{2} = 0) P(X_2 = 0 \mid X_1 = i) \\ &= 3/16 > 0. \end{align*}

$\endgroup$
2
  • $\begingroup$ Isnt the C-K equation $ \begin{align*} P(X_{n+m} = k \mid X_{0} = i) &= \sum_k P(X_{n+m} = j, X_m=k) \mid X_{0} = i) \end{align*} $ ? $\endgroup$ Jan 28 '19 at 11:23
  • 1
    $\begingroup$ They are equivalent :) $\endgroup$
    – Taylor
    Jan 28 '19 at 13:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.