Suppose the weather condition classified as "sunny" (S) or "rainy" (R). Assume the condition on day $n + 1$ is dependent on the days $n$ and $n-1$ only. Work out the transition matrix from the table below.

$$\pmatrix{\mathrm{Yesterday} & \mathrm{Today} & \mathrm{Tomorrow} \\ R & R & R \\ S & S & S \\S & R & R \\R & S & S \\}$$

So far, what I have done is said that if we let $X_n$ be the weather on day $n$, then $X_n$ isn't a discrete time markov chain as it doesn't satify the markov property. To turn it into a DTMC, lets say that $Y_n = (X_{n-1}, X_n)$. This is now a DTMC, with the state space $S = \{1, 2,3, 4\}$ where $1 = (R,R), 2 = (R, S), 3 = (S, R), 4 = (S, S)$. Now I want to work out the transition probabilites.

So first lets calculate $P_{11}$. So this is the probability of going from state 1 to state 1, i.e if it rains yesterday and today, what is the probability it will rain tomorrow. So I am working out

$$P\{Y_{n+1} = 1 | Y_n = 1\} = P\{X_{n} = R, X_{n+1} = R | X_{n-1} = R, X_{n} = R\}$$

I then said from here, using the law of total probability, we get

$$\frac{P\{X_{n} = R, X_{n+1} = R, X_{n-1} = R, X_{n} = R\}}{P\{X_{n-1} = R, X_{n} = R\}}$$

But in the answers, it says this line should read

$$\frac{P\{X_{n} = R, X_{n+1} = R, X_{n-1} = R\}}{P\{X_{n-1} = R, X_{n} = R\}}$$

Then for some reason they go from here to

$$P\{X_{n+1} = R | X_{n-1} = R, X_{n} = R\}$$

First, where does the $X_n$ go from the line above this, and secondly, how do they make the jump to the last line?

EDIT: Ok, this is the data in the table

$$\begin{matrix} \mathrm{\underline{Yesterday}} & \mathrm{\underline{Today}} & \mathrm{\underline{Tomorrow}} & \mathrm{\underline{Prob.}} \\ R & R & R & 0.6 \\ S & S & S & 0.8 \\ S & R & R & 0.5 \\ R & S & S & 0.75 \end{matrix}$$

I tried to work out $P_{12}$ and got my formula to be $P(X_{n+1} = S | X_n = R, X_{n-1} R)$. How would I solve this using the table?

Also, in the answers, they say $P_{13} = 0$, why would this be?

  • 1
    $\begingroup$ Look closely. What is the difference between your formula and the first formula from the answers? Can you deduce why the numerators in both formulas actually express the same thing? $\endgroup$ – S. Kolassa - Reinstate Monica Jan 20 '13 at 13:15
  • $\begingroup$ Because there's two $N_n = R$? and we only need to count it once? $\endgroup$ – Kaish Jan 20 '13 at 13:33
  • $\begingroup$ Should $X_n = R$. Also, I figured out how they get the last bit. Am I correct in why the numerator changes though? $\endgroup$ – Kaish Jan 20 '13 at 13:41
  • $\begingroup$ Exactly. Simply put, the probability that $X_n=R$ is of course equal to the probability that $X_n=R$ and $X_n=R$ (because we are simply saying the same thing twice). And the same holds true if we add additional conditionals to both expressions. $\endgroup$ – S. Kolassa - Reinstate Monica Jan 20 '13 at 13:53
  • 1
    $\begingroup$ ... and there you are! $\endgroup$ – S. Kolassa - Reinstate Monica Jan 20 '13 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.