I want to prove that a mixture of conjugate priors is itself conjugate. It does not look difficult, but I'm still a bit unsure when manipulating probabilities, especially in a Bayesian context. Is this proof correct?

Consider the following prior. $$ p(\theta) = \sum_k p(z=k)p(\theta | z=k) $$ The posterior after observing $D$ is $$ p(\theta | D) = \frac{p(\theta, D)}{p(D)} $$ By the law of total probability $$ \frac{p(\theta, D)}{p(D)} = \frac{\sum_k p(z=k)p(\theta ,D |z=k)}{p(D)} $$ which by the chain rule of probability equals $$ \frac{\sum_k p(z=k) p(D |z=k) p(\theta | D,z=k)}{p(D)} $$ Since $\frac{p(z=k) p(D|z=k)}{p(D)} = p(z=k|D)$ $$ p(\theta | D) = \sum_k p(z=k|D) p(\theta | D, z=k) \mbox{ Q.E.D.} $$

  • 1
    $\begingroup$ Looks okay to me. Why don't you try to write a program for a dataset. But it looks fine. $\endgroup$ – Coder Aug 16 '16 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.