0
$\begingroup$

wiki uses this example to illustrate Markov chains.

The probabilities of weather conditions (modeled as either rainy or sunny), given the weather on the preceding day, can be represented by a transition matrix:

${\displaystyle P={\begin{bmatrix}0.9&0.1\\0.5&0.5\end{bmatrix}}}$

The matrix P represents the weather model in which a sunny day is 90% likely to be followed by another sunny day, and a rainy day is 50% likely to be followed by another rainy day. The columns can be labelled "sunny" and "rainy", and the rows can be labelled in the same order.

The weather on day 1 is known to be sunny. This is represented by a vector in which the "sunny" entry is 100%, and the "rainy" entry is 0%:

${\displaystyle \mathbf {x} ^{(0)}={\begin{bmatrix}1&0\end{bmatrix}}}$

for day n + 1(Note: the original value on wiki is n, which seems to be incorrect)

${\mathbf {x}}^{{(n)}}={\mathbf {x}}^{{(0)}}P^{n}$

The superscript (n) is an index, and not an exponent.

is $P^n$ here the n-Step Transition Probabilities?

$\endgroup$
0
$\begingroup$

Yes, $P^n$ represents the $n$-step transition probability matrix. Here is one way to show that. According to Chapman-Kolmogorov equations, the $(n+m)$-step transition probabilities can be expressed as the sum of products of $n$-step and $m$-step probabilities,as follows:

$$ p_{ij}^{(n+m)}=\sum_{l}p_{il}^{(n)}p_{lj}^{(m)} $$

or, using matrix notation as $P^{(n+m)}$=$P^{(n)}P^{(m)}$. Substituting in the last expression $n-1$ for $n$ and $1$ for $m$ we get $P^{(n-1+1)}$=$P^{(n-1)}P^1$, that is $P^{(n)}$=$P^{(n-1)}P^1$. Applying recursion to the right side, we arrive at

$$P^{(n)}=P^n$$

in other words, the $n$-step transition matrix is the n-th power of the 1-step transition matrix.

$\endgroup$
  • $\begingroup$ Thanks, your answer is very helpful. Given day1 is non-probabilistic, consider day2, is the n of "𝑛-step" equal to 1? $\endgroup$ – fu DL Aug 6 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.