Posterior of binomial and mixed prior

Question

I'm currently studying posterior distribution with likelihood $y|\theta \sim B(n,\theta)$ and mixture of prior distribution $\theta \sim \pi Beta(\alpha_1, \beta_1) + (1-\pi)Beta(\alpha_2, \beta_2)$. The posterior distribution is as follows:

$$ \begin{aligned} p(\theta|y) & \propto \theta^y (1-\theta)^{n-y} \{\pi\frac{\Gamma(\alpha_1 + \beta_1)}{\Gamma(\alpha_1)\Gamma(\beta_1)}\theta^{\alpha_1 -1}(1-\theta)^{\beta_1 - 1} + (1-\pi)\frac{\Gamma(\alpha_2 + \beta_2)}{\Gamma(\alpha_2)\Gamma(\beta_2)}\theta^{\alpha_2 -1}(1-\theta)^{\beta_2 - 1}\} \\ & = \pi\frac{\Gamma(\alpha_1 + \beta_1)}{\Gamma(\alpha_1)\Gamma(\beta_1)}\theta^{y + \alpha_1 -1}(1-\theta)^{n-y +\beta_1 - 1} + (1-\pi)\frac{\Gamma(\alpha_2 + \beta_2)}{\Gamma(\alpha_2)\Gamma(\beta_2)}\theta^{y + \alpha_2 -1}(1-\theta)^{n-y + \beta_2 - 1} \\ \end{aligned} $$

Now, I have to show $p(\theta|y)$ is another mixed beta distribution; that is, $$p(\theta|y) \propto w*Beta(y+\alpha_1, n -y + \beta_1) + (1-w)*Beta(y+\alpha_2, n - y + \beta_2)$$ But, I'm currently stuck to here. How can I induce this formula from the above relationship? Thank you.

Answer 1

If you follow the same reasoning as in the answer you mentioned (https://stats.stackexchange.com/a/100732/271601), you will obtain:

$$ p(\theta|y) \propto \pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}B(\theta;y+\alpha_1,n-y+\beta_1) + (1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}B(\theta;y+\alpha_2,n-y+\beta_2) $$

where $B(\cdot,\cdot)$ refers to the Beta function and $B(\cdot;\cdot,\cdot)$ refers to the Beta distribution. That is to say, the posterior $p(\theta|y)$ is equal to the above expression, up to a multiplicative constant. Since the pseudo weights $\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}$ and $(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}$ do not add up to 1, we can normalize them (i.e. divide both of them by their respective sum), so that their sum effectively adds up to 1. Hence, the new weights on your mixture will be

$$ w_1=\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)} \left(\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}+(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\right)^{-1} $$

and

$$ w_2=(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\left(\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}+(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\right)^{-1} $$

(i.e. the above pseudo weights, but normalized by their sum). You can now write

$$ p(\theta|y) = w_1 B(\theta;y+\alpha_1,n-y+\beta_1) + w_2 B(\theta;y+\alpha_2,n-y+\beta_2) $$

(which is an equality, and not a proportionality relation), with $w_1 + w_2 = 1$.

Answer 2

Imagine an unobserved variable $Z\in\{1,2\}$ that tells you which component of the mixture to use. Your question is a bit ambiguous as to whether $Z$ is chosen once or chosen independently $n$ times, but your computed posterior fits with 'chosen once'.

$Y|Z=1$ will have a Beta$(\alpha_1,\beta_1)$ distribution and $Y|Z=2$ will have a Beta$(\alpha_2,\beta_2)$ distribution. So $$(\alpha_1,\beta_1|Z=1,Y)\sim Beta(\alpha_1+y,\beta_1+n-y)$$ and $$(\alpha_2,\beta_2|Z=1,Y)\sim Beta(\alpha_2,\beta_2)$$ and similarly, mutatis mutandis for $Z=2$.

That is, the posterior is the same sort of mixture as the prior. The mixture components are updated using the formulas for conjugate priors. We also need to update the probability $\pi$ to get the new component probabilities $w$. This is slightly more complicated, but you aren't asking for an explicit formula so that's ok. [Basically, $\pi$ will be updated using the likelihood ratio for getting the observed $y$ from component 2 vs component 1]