# Bayesian inference of parameter governing Markov transition matrix

A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form

$$\begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) & (1-a)(1-b) + ab & a(1-b) \\ b^2 & 2b(1-b) & (1-b)^2 \end{pmatrix}$$

with $a$ and $b$ assigned a uniform prior distribution between 0 and 1.

Let $c_{ij}$ denote the number of transitions from state $i$ to $j$ in the observed chain $X$. I'm interested in sampling from the posterior distribution $$p(a,b|X) \propto p(X|a, b)p(a,b) = \prod_{i,j} {P_{ij}}^{c_{ij}}$$

This can be further developed as $$(1-a)^{2c_{11}}\ (2a(1-a))^{c_{12}}\ a^{2c_{13}}\ (b(1-a))^{c_{21}}\ ((1-a)(1-b) + ab)^{c_{22}}\ (a(1-b))^{c_{23}}\ b^{2c_{31}}\ (2b(1-b))^{c_{32}}\ (1-b)^{2c_{33}}$$ but I don't know if it can be factored into a much nicer form. I'm wondering if there is any way to sample from this distribution without resorting to simulation methods such as MCMC.

EDIT: improved question

• The first step would be to construct the likelihood for an observed chain. Given a fixed p and q, what is the probability of an observed chain? – aleshing Jul 14 '18 at 5:28
• Given the transition matrix $P$, one way of expressing the likelihood would be $\prod_{i=1}^{N-1} P_{X_i, X_{i+1}}$. – Gonzalo Benegas Jul 14 '18 at 5:42
• Ok now write that out in terms of $p$ and $q$ (hint: using indicator functions or counts of specific transitions observed is the best way to go) – aleshing Jul 14 '18 at 14:11
• Another hint: suppose $c_{22}=0$, and simplify the likelihood in this case. Using your independent uniform priors you should be able to find the posterior in closed form. – aleshing Jul 16 '18 at 21:15
• In that case it seems $a$ and $b$ would have beta posterior distributions. But if the assumption does not hold? Do you suggest anything else? – Gonzalo Benegas Jul 16 '18 at 22:11