3
$\begingroup$

In the step of learning Markov chains, I came across few questions from assignments on UTDallas website. Finding the transition probabilities seems a bit hard for me in this one particular question only. The question is

(Sec 7.3, page 355, #1) Consider a system with two components. We observe the state of the system every hour. A given component operating at time $n$ has probability $p$ of failing before the next observation at time $n + 1$. A component that was in a failed condition at time $n$ has a probability $r$ of being repaired by time $n + 1$, independent of how long the component has been in a failed state. The component failures and repairs are mutually independent events. Let $X_n$ be the number of components in operation at time $n$. The process $\{X_n, n = 0, 1, . . .\}$ is a discrete time homogeneous Markov chain with state space $I = \{0, 1, 2\}$.

a) Determine its transition probability matrix, and draw the state diagram.

b) Obtain the steady state probability vector, if it exists

Although the answers are given, but I cannot understand that on what basis the transition probabilities are calculated. Can someone help me in this...

I had the following guesses my ownself (which mostly proved to be wrong)

\begin{bmatrix} 1-r-r^2 & r & r^2\\ ??& ?? & r(1-p) \\ p^2 & ?? & ?? \end{bmatrix}

The actual solution that the website pose is following...

\begin{bmatrix} (1-r)^2 & 2r(1-r) & r^2\\ p(1-r)& pr + (1-p)(1-r) & r(1-p) \\ p^2 & 2p(1-p) & (1-p)^2 \end{bmatrix}

$\endgroup$
  • $\begingroup$ Thanks Yes it is self-study question. What I am asking is not the solution since I am not taking this exam, I am trying to learn the probability, in general, and markov chains, in specific. Appreciate your comment @Tavrock. $\endgroup$ – Kashan Feb 25 '17 at 4:11
  • $\begingroup$ I am looking if someone can help me to learn HERE how transition probabilities are calculated...Since to my understanding, transitional probabilities are different than normal probabilities... $\endgroup$ – Kashan Feb 25 '17 at 10:59
  • $\begingroup$ Can you explain the reasoning behind your guesses? (Also, based on the last comment, do you know what a transition probability is?) $\endgroup$ – Juho Kokkala Feb 25 '17 at 14:27
2
$\begingroup$

Note that, there are two components and each of them individually could be either in working or failure state. Then the states of the Markov chain be designated as $ff$ both components failed, $wf$ one component working and one component failed, and $ww$ both components working. Given that, the components function independently of each other.

  • If the chain is in state $ff$ at time $n$, at the next time point it could be in the state

    • $ff$ with probability $(1-r)(1-r)=(1-r)^2$, as the probability of not being repaired by next time point is $(1-r)$;

    • $wf$ with probability $2r(1-r)$, as it may be the first component
      that was repaired or the the second component that was repaired, so
      that only one of the two components will be working;

    • $ww$ with probability $r^2$, if both the components got repaired.

  • If the chain is in the state $wf$ at time $n$, then at the next time point it could be in the state

    • $ff$ with probability $p(1-r)$, as the working component failed and the component requiring repair was not yet repaired;

    • $wf$ with probability $pr+(1-p)(1-r)$, as the working component was
      not failed and the component requiring repair is not repaired, so
      that only one component is working, which has a probability
      $(1-p)(1-r)$; or the component working at the previous time has
      failed and the component requiring repair has got repaired, which has a probability $pr$; mutually exclusiveness of the events results in the required probability;

    • $ww$ with probability $r(1-p)$, as the working component does not
      require repair and the failed component got repaired.

  • If the chain is in the state $ww$ at time $n$, then at the next time point it could be in the state

    • $ff$ with probability $p^2$, as both components have failed;
    • $wf$ with probability $2p(1-p)$, which is the sum of probabilities
      of two mutually exclusive events, viz., the first component failed
      and the second working or the first component working and the
      second component failed;

      • $ww$ with probability $(1-p)^2$, as both components are still
        working.

Hence, the transition matrix:

\begin{equation*} P=\begin{array}{c|ccc} &ff & wf & ww\\ \hline ff & (1-r)^2& 2r(1-r) & r^2\\ wf & p(1-r) & pr+(1-p)(1-r) & r(1-p) \\ ww & p^2 & 2p(1-p) & (1-p)^2 \end{array} \end{equation*}

$\endgroup$
  • $\begingroup$ Thanks.... Helped me a lot in developing understanding and brushing off some dust over my mind $\endgroup$ – Kashan Feb 26 '17 at 2:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.