Linear combination of conjugate prior

Question

Let's say we want to find the posterior distribution for $\Theta$, where the likelihood model $X|\Theta$ ~ $Binom(8000, \Theta)$. Suppose instead of one distribution for the prior, we use a linear combination. For example $$f_\Theta(\theta) = \frac{1}{3}Beta(10, 1)+\frac{1}{3}Beta(1, 1)+\frac{1}{3}Beta(1, 10)$$

Since the beta distribution is the conjugate prior for the probability of the binomial distribution, the posterior I got is $$P_{\Theta|X}(\theta|x) = w_1*Beta(10+x, 8001-x)+w_2*Beta(1+x, 8001-x)+w_3*Beta(1+x, 8010-x)$$

It seems to make sense that a linear combination of priors results in a linear combination of posteriors, however, I am not sure how to update the weights. Intuitively, the more "likely" posteriors should have a larger weight. Is my approach on the right track?

Answer 1

You are correct to think that the (mixture) weights need updating. I'm going to answer your question a bit more generally than it was asked (and I'm going to change the notation in order to do that).

The problem can be expressed in terms of Bayes rule: \begin{equation} p(\theta|x) = \frac{p(x|\theta)\,p(\theta)}{p(x)} , \end{equation} where \begin{equation} p(x) = \int p(x|\theta)\,p(\theta)\,d\theta . \end{equation}

The thing of interest here is that the prior is a mixture \begin{equation} p(\theta) = \sum_j w_j\, p_j(\theta) . \end{equation} Given the mixture prior, here is some useful notation: \begin{equation} p_j(\theta|x) = \frac{p(x|\theta)\,p_j(\theta)}{p_j(x)} , \end{equation} where \begin{equation} p_j(x) = \int p(x|\theta)\,p_j(\theta)\,d\theta . \end{equation}

We can now express the solution: \begin{equation} p(\theta|x) = \frac{\sum_j w_j\,p(x|\theta)\,p_j(\theta)}{\int \sum_j w_j\,p(x|\theta)\,p_j(\theta)\,d\theta} = \frac{\sum_j w_j\,p_j(\theta|x)\,p_j(x)}{\sum_j w_j\,p_j(x)} = \sum_j \widetilde w_j\,p_j(\theta|x) , \end{equation} where \begin{equation} \widetilde w_j = \frac{w_j\,p_j(x)}{\sum_{j'} w_{j'}\,p_{j'}(x)} . \end{equation} The result is a mixture where the mixture weights have been updated (in addition to the component distributions).