0
$\begingroup$

I am solving the below problem, and facing a few doubts regarding my approach.

Consider a system with two components. We observe the state of the system every hour. A given component operating at time n has probability p of failing before the next observation at time n + 1. A component that was in a failed condition at time n has a probability r of being repaired by time n + 1, independent of how long the component has been in a failed state. The component failures and repairs are mutually independent events. Let $X_n$ be the number of components in operation at time n. The process $\big\{$$X_n$, n = 0, 1, . . .$\big\}$ is a discrete time homogeneous Markov chain with state space $I$ = 0, 1, 2.

  1. Determine its transition probability matrix, and draw the state diagram.
  2. Obtain the steady state probability vector, if it exists.

From my understanding, there are 3 possible states of the system:

  1. All 2 components are working fine
  2. One component has failed and one is working fine
  3. Both components are in the failed state

So I have arrived to the following Markov Chain diagram:

enter image description here

The 2r is because any one system can be repaired if both are in failed state. Using this diagram, I arrived at the below transitional probability matrix:

\begin{bmatrix} -2p & 2p & 0 \\ r & -(r+p) & p \\ 0 & 2r & -2r \end{bmatrix}

Am I thinking in the right direction? And can anybody help me with the second part of the question? To find the steady state probability vector?

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Please add the [self-study] tag & read its wiki. $\endgroup$
    – Silverfish
    Sep 20, 2016 at 8:02

1 Answer 1

Reset to default
0
$\begingroup$

I think that, your are messing up with the state transition probabilities.

I recommend to you to draw the Markov chain with the probabilities of both jumping to a new state and to staying in the current state. Remember that for each state the probability of staying a leaving the state should sum up to $1$.

For example, in the state $0$ (i.e., no component is working):

  • The probability of staying is $p_{00}=(1-r)^2$, that is both components failed to be repaired.
  • The probability of jumping to state $1$ (one components is working) is $p_{01}=2\cdot r(1-r)$, that is, one of the components is repaired and the other is not (note the 2, that accounts for the fact that either of the components can be repaired/no repaired).
  • The probability of jumping to state $2$ (two components are working) from state $0$ is $p_{02}=r^2$ that is, both components are repaired.
  • If you sum up those probabilities you get $r^2 +1+r^2 -2r +2r -2r^2=1$
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks a lot! I can understand now the Markov Chain. But how do we translate this chain in to computing the steady state probability vector? $\endgroup$
    – roynalnaruto
    Sep 20, 2016 at 10:01
  • $\begingroup$ @user2774555 to find the steady state vector you have to solve the following equations: $\pi P =\pi$ and $\sum \pi=1$. Where, $P$ is the transition matrix, and $\pi$ the steady state vector. The constrain $\sum \pi=1$ is to ensure that $\pi$ is a probability vector. This is a system of linear equations and, in your case, can be solved by hand easily. Clearly the seteady state will depend on the value of $r$ and $p$. $\endgroup$
    – PolBM
    Sep 20, 2016 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.