Calculation of transition probabilities of Markov Chain problem

Question

In the step of learning Markov chains, I came across few questions from assignments on UTDallas website. Finding the transition probabilities seems a bit hard for me in this one particular question only. The question is

(Sec 7.3, page 355, #1) Consider a system with two components. We observe the state of the system every hour. A given component operating at time $n$ has probability $p$ of failing before the next observation at time $n + 1$. A component that was in a failed condition at time $n$ has a probability $r$ of being repaired by time $n + 1$, independent of how long the component has been in a failed state. The component failures and repairs are mutually independent events. Let $X_n$ be the number of components in operation at time $n$. The process $\{X_n, n = 0, 1, . . .\}$ is a discrete time homogeneous Markov chain with state space $I = \{0, 1, 2\}$.

a) Determine its transition probability matrix, and draw the state diagram.

b) Obtain the steady state probability vector, if it exists

Although the answers are given, but I cannot understand that on what basis the transition probabilities are calculated. Can someone help me in this...

I had the following guesses my ownself (which mostly proved to be wrong)

\begin{bmatrix} 1-r-r^2 & r & r^2\\ ??& ?? & r(1-p) \\ p^2 & ?? & ?? \end{bmatrix}

The actual solution that the website pose is following...

\begin{bmatrix} (1-r)^2 & 2r(1-r) & r^2\\ p(1-r)& pr + (1-p)(1-r) & r(1-p) \\ p^2 & 2p(1-p) & (1-p)^2 \end{bmatrix}

Answer 1

Note that, there are two components and each of them individually could be either in working or failure state. Then the states of the Markov chain be designated as $ff$ both components failed, $wf$ one component working and one component failed, and $ww$ both components working. Given that, the components function independently of each other.

• If the chain is in state $ff$ at time $n$, at the next time point it could be in the state

  • $ff$ with probability $(1-r)(1-r)=(1-r)^2$, as the probability of not being repaired by next time point is $(1-r)$;

  • $wf$ with probability $2r(1-r)$, as it may be the first component
    that was repaired or the the second component that was repaired, so
    that only one of the two components will be working;

  • $ww$ with probability $r^2$, if both the components got repaired.

• If the chain is in the state $wf$ at time $n$, then at the next time point it could be in the state

  • $ff$ with probability $p(1-r)$, as the working component failed and the component requiring repair was not yet repaired;

  • $wf$ with probability $pr+(1-p)(1-r)$, as the working component was
    not failed and the component requiring repair is not repaired, so
    that only one component is working, which has a probability
    $(1-p)(1-r)$; or the component working at the previous time has
    failed and the component requiring repair has got repaired, which has a probability $pr$; mutually exclusiveness of the events results in the required probability;

  • $ww$ with probability $r(1-p)$, as the working component does not
    require repair and the failed component got repaired.

• If the chain is in the state $ww$ at time $n$, then at the next time point it could be in the state

  • $ff$ with probability $p^2$, as both components have failed;

  • $wf$ with probability $2p(1-p)$, which is the sum of probabilities
    of two mutually exclusive events, viz., the first component failed
    and the second working or the first component working and the
    second component failed;

    • $ww$ with probability $(1-p)^2$, as both components are still
      working.

Hence, the transition matrix:

\begin{equation*} P=\begin{array}{c|ccc} &ff & wf & ww\\ \hline ff & (1-r)^2& 2r(1-r) & r^2\\ wf & p(1-r) & pr+(1-p)(1-r) & r(1-p) \\ ww & p^2 & 2p(1-p) & (1-p)^2 \end{array} \end{equation*}