# Proposed transition matrix for MCMC in two-state Markov Chain

Suppose we would like to model the weather (either sunny $$S$$ or cloudy $$C$$) using a two-state Markov Chain, given a set of data collected from 10000 days: $$CCCSSSSSSCCCSSSSSSCCCC...$$

We can use the metropolis-hastings algorithm to get a target transition matrix $$P$$ whose invariant distribution is given by $$\pi = \left(\frac{\text{# of S's from the data}}{10000}, \frac{\text{# of C's from the data}}{10000}\right)$$ from the data.

Now how do we choose the proposed transition matrix $$Q$$? Naturally, we don't want to choose $$Q$$ to be $$Q = \begin{pmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \\ \end{pmatrix}$$ because then a typical sample from this $$P$$ would look like $$CCSCSSCSCSCCSCSCCCSCCS...$$ (which is unrealistic for modeling the weather) instead of our data $$CCCSSSSSSCCCSSSSSSCCCC...$$

Is there a systematic way of picking the $$Q$$ given the data set like above? I know the intuition would be say let
$$Q = \begin{pmatrix} 0.9 & 0.1 \\ 0.1 & 0.9 \\ \end{pmatrix}$$ but the $$0.9$$ and $$0.1$$ are still guesses.

• You can't accomplish this with just two states. Use more. See stats.stackexchange.com/questions/115883 and stats.stackexchange.com/questions/32531 for details.
– whuber
Feb 2, 2022 at 0:04
• @whuber In case we have say 3 states ( S, C, and Rain), given a set of data, would we be able to pick the proposal matrix $Q$ in some "optimal" way? Edit: just saw the link, I will take a look, thanks!
– Xiao
Feb 2, 2022 at 0:09
• You don't pick the matrix: you estimate it from the observed frequencies. The states you need correspond to sequences of C and S, not additional types of weather! These sequences are called "n-grams" ($n$ is the length).
– whuber
Feb 2, 2022 at 0:11
• @whuber I guess we would have to use state space like $\{SSSS, SSSC, SSCC, ...\}$, right?
– Xiao
Feb 2, 2022 at 0:40
• You could use a state space of $(n_s, n_c)$, where the indices refer to the number of consecutive observations in each state; obviously one of the two will always equal zero. Then your transition matrix is of size at least equal to the sum of the two maximum sequence lengths. Feb 2, 2022 at 1:34

In the current setting, if the transition matrix $$P$$ is the quantity of interest and is inferred from the data at hand, an MCMC method such as Metropolis-Hastings would operate on the space of such transitions matrices, provided a prior distribution on the $$P$$'s and a sampling distribution on the data, indexed by $$P$$, are given .