A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form

$$ \begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) & (1-a)(1-b) + ab & a(1-b) \\ b^2 & 2b(1-b) & (1-b)^2 \end{pmatrix} $$

with $a$ and $b$ assigned a uniform prior distribution between 0 and 1.

Let $c_{ij}$ denote the number of transitions from state $i$ to $j$ in the observed chain $X$. I'm interested in sampling from the posterior distribution $$ p(a,b|X) \propto p(X|a, b)p(a,b) = \prod_{i,j} {P_{ij}}^{c_{ij}} $$

This can be further developed as $$ (1-a)^{2c_{11}}\ (2a(1-a))^{c_{12}}\ a^{2c_{13}}\ (b(1-a))^{c_{21}}\ ((1-a)(1-b) + ab)^{c_{22}}\ (a(1-b))^{c_{23}}\ b^{2c_{31}}\ (2b(1-b))^{c_{32}}\ (1-b)^{2c_{33}} $$ but I don't know if it can be factored into a much nicer form. I'm wondering if there is any way to sample from this distribution without resorting to simulation methods such as MCMC.

EDIT: improved question

  • $\begingroup$ The first step would be to construct the likelihood for an observed chain. Given a fixed p and q, what is the probability of an observed chain? $\endgroup$ – aleshing Jul 14 '18 at 5:28
  • $\begingroup$ Given the transition matrix $P$, one way of expressing the likelihood would be $\prod_{i=1}^{N-1} P_{X_i, X_{i+1}}$. $\endgroup$ – Gonzalo Benegas Jul 14 '18 at 5:42
  • $\begingroup$ Ok now write that out in terms of $p$ and $q$ (hint: using indicator functions or counts of specific transitions observed is the best way to go) $\endgroup$ – aleshing Jul 14 '18 at 14:11
  • $\begingroup$ Another hint: suppose $c_{22}=0$, and simplify the likelihood in this case. Using your independent uniform priors you should be able to find the posterior in closed form. $\endgroup$ – aleshing Jul 16 '18 at 21:15
  • $\begingroup$ In that case it seems $a$ and $b$ would have beta posterior distributions. But if the assumption does not hold? Do you suggest anything else? $\endgroup$ – Gonzalo Benegas Jul 16 '18 at 22:11

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.