Suppose we would like to model the weather (either sunny $S$ or cloudy $C$) using a two-state Markov Chain, given a set of data collected from 10000 days: $$CCCSSSSSSCCCSSSSSSCCCC...$$
We can use the metropolis-hastings algorithm to get a target transition matrix $P$ whose invariant distribution is given by $$\pi = \left(\frac{\text{# of $S$'s from the data}}{10000}, \frac{\text{# of $C$'s from the data}}{10000}\right)$$ from the data.
Now how do we choose the proposed transition matrix $Q$? Naturally, we don't want to choose $Q$ to be $$Q = \begin{pmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \\ \end{pmatrix}$$ because then a typical sample from this $P$ would look like $$CCSCSSCSCSCCSCSCCCSCCS...$$ (which is unrealistic for modeling the weather) instead of our data $$CCCSSSSSSCCCSSSSSSCCCC...$$
Is there a systematic way of picking the $Q$ given the data set like above? I know the intuition would be say let
$$Q = \begin{pmatrix}
0.9 & 0.1 \\
0.1 & 0.9 \\
\end{pmatrix}$$
but the $0.9$ and $0.1$ are still guesses.
C
andS
, not additional types of weather! These sequences are called "n-grams" ($n$ is the length). $\endgroup$