# Proposed transition matrix for MCMC in two-state Markov Chain

Suppose we would like to model the weather (either sunny $$S$$ or cloudy $$C$$) using a two-state Markov Chain, given a set of data collected from 10000 days: $$CCCSSSSSSCCCSSSSSSCCCC...$$

We can use the metropolis-hastings algorithm to get a target transition matrix $$P$$ whose invariant distribution is given by $$\pi = \left(\frac{\text{# of S's from the data}}{10000}, \frac{\text{# of C's from the data}}{10000}\right)$$ from the data.

Now how do we choose the proposed transition matrix $$Q$$? Naturally, we don't want to choose $$Q$$ to be $$Q = \begin{pmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \\ \end{pmatrix}$$ because then a typical sample from this $$P$$ would look like $$CCSCSSCSCSCCSCSCCCSCCS...$$ (which is unrealistic for modeling the weather) instead of our data $$CCCSSSSSSCCCSSSSSSCCCC...$$

Is there a systematic way of picking the $$Q$$ given the data set like above? I know the intuition would be say let
$$Q = \begin{pmatrix} 0.9 & 0.1 \\ 0.1 & 0.9 \\ \end{pmatrix}$$ but the $$0.9$$ and $$0.1$$ are still guesses.

• You can't accomplish this with just two states. Use more. See stats.stackexchange.com/questions/115883 and stats.stackexchange.com/questions/32531 for details.
– whuber
Feb 2 at 0:04
• @whuber In case we have say 3 states ( S, C, and Rain), given a set of data, would we be able to pick the proposal matrix $Q$ in some "optimal" way? Edit: just saw the link, I will take a look, thanks!
– Xiao
Feb 2 at 0:09
• You don't pick the matrix: you estimate it from the observed frequencies. The states you need correspond to sequences of C and S, not additional types of weather! These sequences are called "n-grams" ($n$ is the length).
– whuber
Feb 2 at 0:11
• @whuber I guess we would have to use state space like $\{SSSS, SSSC, SSCC, ...\}$, right?
– Xiao
Feb 2 at 0:40
• You could use a state space of $(n_s, n_c)$, where the indices refer to the number of consecutive observations in each state; obviously one of the two will always equal zero. Then your transition matrix is of size at least equal to the sum of the two maximum sequence lengths. Feb 2 at 1:34

In the current setting, if the transition matrix $$P$$ is the quantity of interest and is inferred from the data at hand, an MCMC method such as Metropolis-Hastings would operate on the space of such transitions matrices, provided a prior distribution on the $$P$$'s and a sampling distribution on the data, indexed by $$P$$, are given .