0
$\begingroup$

A markov chain model TPM for daily rainfall is given by:

[0.4 0.6 0.0;
 0.2 0.6 0.2;
 0.0 0.7 0.3];

Thus, it is not possible to pass directly from state 1 to state 3 or state 3 to state 1 without going to state 2. How would I compute the fraction of time the system is in each of the states?

$\endgroup$
1
  • $\begingroup$ Add self-study tag? What have you tried? Suppose you had this vector of "fractions of time" $\pi$, what property does it have? $\endgroup$
    – Neil G
    Jun 2 '15 at 23:15
2
$\begingroup$

Under certain conditions, including the chain being irreducible (all states are accessible) and the states being all positive recurrent (they will be reached with probability 1), the Markov chain will have a stationary distribution in the limit.

Let $$ \mathbf{P} = \left[\matrix{0.4 & .06 & 0.0\\ 0.2 & 0.6 & 0.2\\ 0.0 & 0.7 & 0.3}\right] $$

Remember that $\mathbf{P}$ is the probability of a one-step move. $\mathbf{P}^2$ is the probability of where you end up after two steps. In this case:

$$ \mathbf{P}^2 = \mathbf{P}\times\mathbf{P} = \left[\matrix{0.28 &0.60 &0.12\\ 0.20 &0.62 &0.18\\ 0.14 &0.63 &0.23}\right] $$

So you see that all states are reachable from all others, although not in one jump. Therefore, the limiting distribution is $\mathbf{P}^\infty$, which can be estimated be repeated applications of matrix multiplication or calculated through other techniques (see here for an example).

In this case, we get

$$ \mathbf{P}^\infty \approx \left[\matrix{0.2058824 &0.6176471 &0.1764706\\ 0.2058824 &0.6176471 &0.1764706\\ 0.2058824 &0.6176471 &0.1764706}\right] $$

Letting $\pi$ represent the stationary distribution, or the fraction of time spent in each state, we have: $$ \pi \approx (20.6\%, 61.8\%, 17.6\%) $$

As one has to go through state two to get to either state 1 or state 3 in any given step, the magnitude of state 2 with respect to states 1 and 3 is unsurprising.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.