1
$\begingroup$

I'm currently studying posterior distribution with likelihood $y|\theta \sim B(n,\theta)$ and mixture of prior distribution $\theta \sim \pi Beta(\alpha_1, \beta_1) + (1-\pi)Beta(\alpha_2, \beta_2)$. The posterior distribution is as follows:

$$ \begin{aligned} p(\theta|y) & \propto \theta^y (1-\theta)^{n-y} \{\pi\frac{\Gamma(\alpha_1 + \beta_1)}{\Gamma(\alpha_1)\Gamma(\beta_1)}\theta^{\alpha_1 -1}(1-\theta)^{\beta_1 - 1} + (1-\pi)\frac{\Gamma(\alpha_2 + \beta_2)}{\Gamma(\alpha_2)\Gamma(\beta_2)}\theta^{\alpha_2 -1}(1-\theta)^{\beta_2 - 1}\} \\ & = \pi\frac{\Gamma(\alpha_1 + \beta_1)}{\Gamma(\alpha_1)\Gamma(\beta_1)}\theta^{y + \alpha_1 -1}(1-\theta)^{n-y +\beta_1 - 1} + (1-\pi)\frac{\Gamma(\alpha_2 + \beta_2)}{\Gamma(\alpha_2)\Gamma(\beta_2)}\theta^{y + \alpha_2 -1}(1-\theta)^{n-y + \beta_2 - 1} \\ \end{aligned} $$

Now, I have to show $p(\theta|y)$ is another mixed beta distribution; that is, $$p(\theta|y) \propto w*Beta(y+\alpha_1, n -y + \beta_1) + (1-w)*Beta(y+\alpha_2, n - y + \beta_2)$$ But, I'm currently stuck to here. How can I induce this formula from the above relationship? Thank you.

$\endgroup$

2 Answers 2

1
$\begingroup$

If you follow the same reasoning as in the answer you mentioned (https://stats.stackexchange.com/a/100732/271601), you will obtain:

$$ p(\theta|y) \propto \pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}B(\theta;y+\alpha_1,n-y+\beta_1) + (1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}B(\theta;y+\alpha_2,n-y+\beta_2) $$

where $B(\cdot,\cdot)$ refers to the Beta function and $B(\cdot;\cdot,\cdot)$ refers to the Beta distribution. That is to say, the posterior $p(\theta|y)$ is equal to the above expression, up to a multiplicative constant. Since the pseudo weights $\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}$ and $(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}$ do not add up to 1, we can normalize them (i.e. divide both of them by their respective sum), so that their sum effectively adds up to 1. Hence, the new weights on your mixture will be

$$ w_1=\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)} \left(\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}+(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\right)^{-1} $$

and

$$ w_2=(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\left(\pi \frac{B(y+\alpha_1,n-y+\beta_1)}{B(\alpha_1,\beta_1)}+(1-\pi) \frac{B(y+\alpha_2,n-y+\beta_2)}{B(\alpha_2,\beta_2)}\right)^{-1} $$

(i.e. the above pseudo weights, but normalized by their sum). You can now write

$$ p(\theta|y) = w_1 B(\theta;y+\alpha_1,n-y+\beta_1) + w_2 B(\theta;y+\alpha_2,n-y+\beta_2) $$

(which is an equality, and not a proportionality relation), with $w_1 + w_2 = 1$.

$\endgroup$
1
$\begingroup$

Imagine an unobserved variable $Z\in\{1,2\}$ that tells you which component of the mixture to use. Your question is a bit ambiguous as to whether $Z$ is chosen once or chosen independently $n$ times, but your computed posterior fits with 'chosen once'.

$Y|Z=1$ will have a Beta$(\alpha_1,\beta_1)$ distribution and $Y|Z=2$ will have a Beta$(\alpha_2,\beta_2)$ distribution. So $$(\alpha_1,\beta_1|Z=1,Y)\sim Beta(\alpha_1+y,\beta_1+n-y)$$ and $$(\alpha_2,\beta_2|Z=1,Y)\sim Beta(\alpha_2,\beta_2)$$ and similarly, mutatis mutandis for $Z=2$.

That is, the posterior is the same sort of mixture as the prior. The mixture components are updated using the formulas for conjugate priors. We also need to update the probability $\pi$ to get the new component probabilities $w$. This is slightly more complicated, but you aren't asking for an explicit formula so that's ok. [Basically, $\pi$ will be updated using the likelihood ratio for getting the observed $y$ from component 2 vs component 1]

$\endgroup$
1
  • $\begingroup$ Thank you. Then, could you explain a bit more about how to update the $\pi$ to obtain $w$? Now I see that I was wondering how to calculate the $w$ from all the $\Gamma$ stuff in the above expressions. For reference, I was looking at this site stats.stackexchange.com/a/100732/372718 to generalize the expression $p(\theta|y)$. $\endgroup$
    – jason 1
    Sep 25, 2023 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.