So I have a mixture prior of:

$p(\mu) = \pi p_0(\mu \mid \mu_0, \sigma^2_0) + (1-\pi) p_1(\mu \mid \mu_1, \sigma^2_1)$

and a likelihood of

$$p(y\mid\mu) = (2\pi\sigma^2)^{-n/2} \exp\left(\frac{-1}{2\sigma^2}\sum_{i=1}^n(y_i - \mu)^2\right)$$

I've been googling for ages trying to find what the posterior weights are, but I can't find the solution.

The answer I got was that the posterior (unnormalised) weight on $p_0$ was:

$$\pi\frac{\sqrt{2\pi \frac{\sigma^2\sigma^2_0}{\sigma^2 + n\sigma^2_0}}}{\sqrt{2\pi\sigma^2_0}}$$

And the same structure for $p_1$.

I know you can cancel this down, but I wanted to leave it in full to provide more information about where it came from.

What I did was $p(\mu)p(y \mid \mu)$, and then isolated the normal posterior component, and dividing and multiplying by the numerator of the above unnormalised weight.

Hope that makes sense.


So the question is, what is actually the form of the posterior weights? I put the answer I got above, but it doesn't seem right to me based on some JAGS stuff I've done and the fact it seemingly ignores the actual data.

I'm asking for this:

Conjugate mixture of beta distributions - weights

But with Normal distributions everywhere.


A posterior associated with a mixture prior$$\rho \pi_1(\theta) + (1-\rho)\pi_2(\theta)$$writes down as\begin{align}\pi(\theta\mid x)&\propto\rho \pi_1(\theta)f(x\mid\theta) + (1-\rho)\pi_2(\theta)f(x\mid\theta)\\&=\rho m_1(x)\frac{\pi_1(\theta)f(x\mid\theta)}{m_1(x)} + (1-\rho)m_2(x)\frac{\pi_2(\theta)f(x\mid\theta)}{m_2(x)}\end{align} where $m_1(\cdot)$ and $m_2(\cdot)$ are the respective marginals. Therefore the new weights are (proportional to)$$\rho m_1(x)\qquad\text{and}\qquad(1-\rho)m_2(x)$$The marginals are easy to derive: if $$\bar x_n\mid\mu\sim\mathcal{N}(\mu,\sigma^2/n)\qquad\text{and}\qquad\mu\sim\mathcal{N}(\mu_1,\sigma_1^2)$$then$$\bar x_n\sim\mathcal{N}(\mu_1,\sigma_1^2+\sigma^2/n)$$i.e.$$m_1(\bar x_n)=\frac{1}{\sqrt{\sigma_1^2+\sigma^2/n}}\exp\{-(\bar x_n-\mu_1)^2/2(\sigma_1^2+\sigma^2/n)\}$$ This shows why the derivation in

enter image description here

is incorrect, as it forgets about the missing bits in the perfect squares.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.