The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with

$p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is

P =$\begin{pmatrix} \frac{1}{4} & \frac{3}{4} & 0 & 0 & ...\\\ \frac{1}{4} & 0 & \frac{3}{4} & 0 & ... \\\ \frac{1}{4} & 0 & 0 & \frac{3}{4} &... \\\ \vdots & \vdots&\vdots&\vdots& \ddots \end{pmatrix}$

Find the irreducible classes of intercommunicating states. For each class, state:

(a) whether it is transient, positive recurrent or null recurrent (hint - think about the distribution of the return times - say to state 0 - in this case. From there, you can work out whether the states have a finite or infinite expected return time. Can you work out what sort of states you have here?)

(b) its periodicity.

I have tried to approach the first part with a state space diagram and have found that only state 0 and 1 intercommunicate and that the rest of the states do not. But, it is possible to return to every state at some point (say we got to 3, we can then go to state 4, then state 0, 1, 2 and end up at 3 again. So would I put {0,1,2,3,4, ...} into one class?

I guess I am also not 100% sure about how to classify the states.


closed as off-topic by Xi'an, Michael Chernick, Peter Flom Jan 28 at 12:06

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Self-study questions (including textbook exercises, old exam papers, and homework) that seek to understand the concepts are welcome, but those that demand a solution need to indicate clearly at what step help or advice are needed. For help writing a good self-study question, please visit the meta pages." – Xi'an, Michael Chernick, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.


All states communicate by the Chapman-Kolmogorov equation. For any $i,k$

\begin{align*} P(X_n = k \mid X_1 = i) &= \sum_j P(X_n = k \mid X_{2} = j) P(X_2 = j \mid X_1 = i) \\ &\ge P(X_n = k \mid X_{2} = 0) P(X_2 = 0 \mid X_1 = i) \\ &= 3/16 > 0. \end{align*}

  • $\begingroup$ Isnt the C-K equation $ \begin{align*} P(X_{n+m} = k \mid X_{0} = i) &= \sum_k P(X_{n+m} = j, X_m=k) \mid X_{0} = i) \end{align*} $ ? $\endgroup$ – New phone who this Jan 28 at 11:23
  • 1
    $\begingroup$ They are equivalent :) $\endgroup$ – Taylor Jan 28 at 13:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.