# Markov Chain for system with failure and repair

I am solving the below problem, and facing a few doubts regarding my approach.

Consider a system with two components. We observe the state of the system every hour. A given component operating at time n has probability p of failing before the next observation at time n + 1. A component that was in a failed condition at time n has a probability r of being repaired by time n + 1, independent of how long the component has been in a failed state. The component failures and repairs are mutually independent events. Let $X_n$ be the number of components in operation at time n. The process $\big\{$$X_n$, n = 0, 1, . . .$\big\}$ is a discrete time homogeneous Markov chain with state space $I$ = 0, 1, 2.

1. Determine its transition probability matrix, and draw the state diagram.
2. Obtain the steady state probability vector, if it exists.

From my understanding, there are 3 possible states of the system:

1. All 2 components are working fine
2. One component has failed and one is working fine
3. Both components are in the failed state

So I have arrived to the following Markov Chain diagram: The 2r is because any one system can be repaired if both are in failed state. Using this diagram, I arrived at the below transitional probability matrix:

\begin{bmatrix} -2p & 2p & 0 \\ r & -(r+p) & p \\ 0 & 2r & -2r \end{bmatrix}

Am I thinking in the right direction? And can anybody help me with the second part of the question? To find the steady state probability vector?

Thanks!

• Please add the [self-study] tag & read its wiki. – Silverfish Sep 20 '16 at 8:02

## 1 Answer

I think that, your are messing up with the state transition probabilities.

I recommend to you to draw the Markov chain with the probabilities of both jumping to a new state and to staying in the current state. Remember that for each state the probability of staying a leaving the state should sum up to $1$.

For example, in the state $0$ (i.e., no component is working):

• The probability of staying is $p_{00}=(1-r)^2$, that is both components failed to be repaired.
• The probability of jumping to state $1$ (one components is working) is $p_{01}=2\cdot r(1-r)$, that is, one of the components is repaired and the other is not (note the 2, that accounts for the fact that either of the components can be repaired/no repaired).
• The probability of jumping to state $2$ (two components are working) from state $0$ is $p_{02}=r^2$ that is, both components are repaired.
• If you sum up those probabilities you get $r^2 +1+r^2 -2r +2r -2r^2=1$
• Thanks a lot! I can understand now the Markov Chain. But how do we translate this chain in to computing the steady state probability vector? – roynalnaruto Sep 20 '16 at 10:01
• @user2774555 to find the steady state vector you have to solve the following equations: $\pi P =\pi$ and $\sum \pi=1$. Where, $P$ is the transition matrix, and $\pi$ the steady state vector. The constrain $\sum \pi=1$ is to ensure that $\pi$ is a probability vector. This is a system of linear equations and, in your case, can be solved by hand easily. Clearly the seteady state will depend on the value of $r$ and $p$. – PolBM Sep 20 '16 at 10:38