# Linear combination of conjugate prior

Let's say we want to find the posterior distribution for $$\Theta$$, where the likelihood model $$X|\Theta$$ ~ $$Binom(8000, \Theta)$$. Suppose instead of one distribution for the prior, we use a linear combination. For example $$f_\Theta(\theta) = \frac{1}{3}Beta(10, 1)+\frac{1}{3}Beta(1, 1)+\frac{1}{3}Beta(1, 10)$$

Since the beta distribution is the conjugate prior for the probability of the binomial distribution, the posterior I got is $$P_{\Theta|X}(\theta|x) = w_1*Beta(10+x, 8001-x)+w_2*Beta(1+x, 8001-x)+w_3*Beta(1+x, 8010-x)$$

It seems to make sense that a linear combination of priors results in a linear combination of posteriors, however, I am not sure how to update the weights. Intuitively, the more "likely" posteriors should have a larger weight. Is my approach on the right track?

## 1 Answer

You are correct to think that the (mixture) weights need updating. I'm going to answer your question a bit more generally than it was asked (and I'm going to change the notation in order to do that).

The problem can be expressed in terms of Bayes rule: $$$$p(\theta|x) = \frac{p(x|\theta)\,p(\theta)}{p(x)} ,$$$$ where $$$$p(x) = \int p(x|\theta)\,p(\theta)\,d\theta .$$$$

The thing of interest here is that the prior is a mixture $$$$p(\theta) = \sum_j w_j\, p_j(\theta) .$$$$ Given the mixture prior, here is some useful notation: $$$$p_j(\theta|x) = \frac{p(x|\theta)\,p_j(\theta)}{p_j(x)} ,$$$$ where $$$$p_j(x) = \int p(x|\theta)\,p_j(\theta)\,d\theta .$$$$

We can now express the solution: $$$$p(\theta|x) = \frac{\sum_j w_j\,p(x|\theta)\,p_j(\theta)}{\int \sum_j w_j\,p(x|\theta)\,p_j(\theta)\,d\theta} = \frac{\sum_j w_j\,p_j(\theta|x)\,p_j(x)}{\sum_j w_j\,p_j(x)} = \sum_j \widetilde w_j\,p_j(\theta|x) ,$$$$ where $$$$\widetilde w_j = \frac{w_j\,p_j(x)}{\sum_{j'} w_{j'}\,p_{j'}(x)} .$$$$ The result is a mixture where the mixture weights have been updated (in addition to the component distributions).