2
$\begingroup$

Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this:

  • For each object a coin is tossed (outcome $X_i$ = $H$ or $T$).

  • We know the coin is fair, so $X_i \sim \text{Bernoulli}(0.5)$, but we do not observe the outcome $X_i$ of the coin toss.

  • Then dependent on $X_i=x_i$ you observe

    • $Y_i | x_i=H \sim N(\mu_H, \sigma_H)$
    • $Y_i | x_i=T \sim N(\mu_T, \sigma_T)$

Let's assume we have prior information on $\mu_H$, $\sigma_H$, $\mu_T$ and $\sigma_H$, and we'd like to get a Bayesian posterior for them after observing some data.

I think I could treat the $X_i$ as an unobserved latent discrete parameters with a $\text{Bernoulli}(0.5)$ prior for each one of them, right? And one could sample that kind of model with Metropolis-Hastings and/or Gibbs sampling, right (e.g. using JAGS)? Is that the simplest way of approaching this?

On the other hand latent parameters are tricky for Hamiltonian Monte Carlo, so is using e.g. Stan here really hard? Or is there an easy way of summing out the latent discrete parameters? However, I assume it's not as easy as the log-likelihood contribution by each observation being $0.5 \times f_\text{Normal}(y_i | \mu_h, \sigma_H) + 0.5 \times f_\text{Normal}(y_i | \mu_T, \sigma_T)$, or is it (or do I have to write down all possible data constellations of how the $X_i$ and could be and write down the likelihood for each weighted by the probability of each of them?)?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ 1) This is called a Gaussian Mixture Model, and yes Bayes is a great way to do inference on it (the EM algo also works). 2) Yes, HMC alone can't deal with discrete variables (but, contrary to popular belief, no one will stop you from running HMC on the continuous parameters and using standard proposals for the discrete variables within your Gibbs sampler). 3) Yes, integrating out the latent states really is that easy, and no, you don't need to write out all possible constellations of the data so long as you assume independence (which I took your Bernoulli assumption to imply). $\endgroup$
    – John Madden
    Commented Aug 23, 2022 at 15:29
  • $\begingroup$ Oh and I should add the coin need not be fair; it's possible to estimate that probability as well. $\endgroup$
    – John Madden
    Commented Aug 23, 2022 at 15:30

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.