A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form

$$ \begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) & (1-a)(1-b) + ab & a(1-b) \\ b^2 & 2b(1-b) & (1-b)^2 \end{pmatrix} $$

with $a$ and $b$ assigned a uniform prior distribution between 0 and 1.

Let $c_{ij}$ denote the number of transitions from state $i$ to $j$ in the observed chain $X$. I'm interested in sampling from the posterior distribution $$ p(a,b|X) \propto p(X|a, b)p(a,b) = \prod_{i,j} {P_{ij}}^{c_{ij}} $$

This can be further developed as $$ (1-a)^{2c_{11}}\ (2a(1-a))^{c_{12}}\ a^{2c_{13}}\ (b(1-a))^{c_{21}}\ ((1-a)(1-b) + ab)^{c_{22}}\ (a(1-b))^{c_{23}}\ b^{2c_{31}}\ (2b(1-b))^{c_{32}}\ (1-b)^{2c_{33}} $$ but I don't know if it can be factored into a much nicer form. I'm wondering if there is any way to sample from this distribution without resorting to simulation methods such as MCMC.

EDIT: improved question

  • $\begingroup$ The first step would be to construct the likelihood for an observed chain. Given a fixed p and q, what is the probability of an observed chain? $\endgroup$
    – aleshing
    Jul 14 '18 at 5:28
  • $\begingroup$ Given the transition matrix $P$, one way of expressing the likelihood would be $\prod_{i=1}^{N-1} P_{X_i, X_{i+1}}$. $\endgroup$ Jul 14 '18 at 5:42
  • $\begingroup$ Ok now write that out in terms of $p$ and $q$ (hint: using indicator functions or counts of specific transitions observed is the best way to go) $\endgroup$
    – aleshing
    Jul 14 '18 at 14:11
  • $\begingroup$ Another hint: suppose $c_{22}=0$, and simplify the likelihood in this case. Using your independent uniform priors you should be able to find the posterior in closed form. $\endgroup$
    – aleshing
    Jul 16 '18 at 21:15
  • $\begingroup$ In that case it seems $a$ and $b$ would have beta posterior distributions. But if the assumption does not hold? Do you suggest anything else? $\endgroup$ Jul 16 '18 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.