# Need help deriving a gibbs sampler for a normal mixture model with two components

Let $\theta_i$ be an indicator that the i-th eruption is a long eruption. (i.e. $\theta_i = 1$ if the i-th eruption is long and $\theta_i = 0$ otherwise.) Assume the following model and derive a Gibbs sampler for the following normal mixture model with two components:

• $X_i|\theta_i \stackrel{\text{ind}}{\sim} (1-\theta_i) N(\mu_1, \sigma^2_1) + (\theta_i) N(\mu_2, \sigma^2_2)$
• $\theta_i \stackrel{\text{iid}}{\sim}\text{Bernoulli}(\pi_1)$
• $\pi_1 \sim \text{Uniform}(0,1)$
• $\mu_1, \mu_2 \sim N(0,1000)$
• $\sigma^2_1, \sigma^2_2 \sim \text{InverseGamma}(\text{shape} = 0.001, \text{rate} = 0.001)$

Having trouble figuring out how to start, especially since there is an indicator in it

Thank you!

• This looks like a fairly standard exercise you've been set. It should probably carry the self-study tag and should follow the guidelines at the self-tady tag wiki, including some indication of what you've tried and where your problems specifically lie. – Glen_b Dec 11 '14 at 4:08
• I've tried to improve the formatting, but it wasn't 100% clear what you meant by "~ind~" (independent?) so I didn't fix that – Glen_b Dec 11 '14 at 4:12
• Are you sure you have a different $\theta_i$ for each observation? – Zen Dec 14 '14 at 0:07
• @Zen: yes, each observation in X has some probability $\theta_i$ of coming from one of the two normal distributions.,, Anyhelp would be sincerely appreciated. – Vic Cantor Dec 14 '14 at 1:07

I assume this is vanilla Gibbs. You don't say if you're programming from scratch or what. I'll start with some basic hints as if you were writing the whole thing.

The Gibbs sampler relies on sampling from full conditional distributions, so you need to start trying to write down all your conditional posteriors (so you can see how to sample from them)

So can you write down the conditional posterior for say $\left[\mu_1|\mathbf{x,\theta}\right]$?

(If you know $\mathbf{\theta}$, what distribution do you think it should look like?)

Or $\left[\theta_i|\mathbf{X,\mu,...}\right]$?

(What family would you expect the posterior to be from?)

• Thank you Glen_b: I'm precisely stuck on the conditional posteriors. I know mu1|rest and mu2|rest follow normal distribtions based on conjugacy, and sigma^2_1|rest and sigma^2_2|rest follow Innverse Gamma distribitions. I don't know how get the conditional posterior for pi_1 and how the indicator from bernoulli and uniform(0,1) come into play – Vic Cantor Dec 11 '14 at 4:47
• If you know the $\theta_i$, is there any further information about $\pi$ in $X$ or $\mu$? – Glen_b Dec 11 '14 at 4:59
• thanks Glen_b i'm actually stuck determining the conditional posterior for $\theta_i|rest$ – Vic Cantor Dec 13 '14 at 16:46
• Can you calculate the likelihood at $\theta_i=0$ and $\theta_i=1$? Are you sampling the $\theta_i$ one at a time or as a block? – Glen_b Dec 14 '14 at 21:27