1
$\begingroup$

I first read about the Beta PDF in the context that it was conjugate to the Binomial distribution; a Beta prior with a Binomial likelihood returns a Beta posterior. So this sounds to me like a multilevel model. However, I've come across the Beta-Binomial PMF.

Differences I've noticed: Beta-Binomial conj prior is continuous and a closed form MAP solution exists. The Beta-Binomial mixture is discrete and an MLE solution exists.

Beyond these simple observations, I'm not sure when each would/wouldn't be appropriate and how they differ. Thoughts? Is there a deeper relationship here or is it just inconvenient that they have such similar names?

Edit: In response to Tim's answer, I've added the below from my comment on his answer:

Follow up, I see that among PyMC3's discrete likelihood functions Beta-Binomial is built in. Given what you've said. When would I use the Binomial likelihood vs the Beta-Binomial likelihood, both assuming a Beta prior? And how do these differences affect terms of the posterior-predictive distribution?

$\endgroup$

1 Answer 1

4
$\begingroup$

They are both pieces of the beta-binomial model. In beta-binomial model, the predicted variable $y$ follows the binomial distribution, where the number of samples $n$ is known and we want to learn the "probability of success" $p$. For this, we are using a Bayesian model, where beta distribution with hyperparameters $\alpha$ and $\beta$ serves as a prior for $p$:

$$\begin{align} p &\sim \mathsf{Beta}(\alpha, \beta) \\ y &\sim \mathsf{Binomial}(n, p) \end{align}$$

In such a case, due to conjugacy, the posterior distribution for $p$ is also beta distributed. On another hand, the posterior predictive distribution, i.e. the distribution for your predictions of $\hat y$, is beta-binomial distribution.

$\endgroup$
5
  • $\begingroup$ Follow up, I see that among PyMC3's discrete likelihood functions Beta-Binomial (docs.pymc.io/en/stable/api/distributions/…) is built in. Given what you've said. When would I use the Binomial likelihood vs the Beta-Binomial likelihood, both assuming a Beta prior? $\endgroup$
    – jbuddy_13
    Nov 3, 2021 at 15:30
  • 1
    $\begingroup$ @jbuddy_13 The beta-binomial distribution can be used to describe over/under-dispersion. The conjugate prior distribution won't be a beta distribution. I imagine that it will be something complex, already the conjugate distribution of the beta distribution is something difficult: stats.stackexchange.com/a/67511 $\endgroup$ Nov 3, 2021 at 16:45
  • $\begingroup$ @SextusEmpiricus & Tim, I've made some rather large changes to this post in reference to both of your insightful comments. Curious on your thoughts about inference and inconsistency of parameters... $\endgroup$
    – jbuddy_13
    Nov 3, 2021 at 18:21
  • 1
    $\begingroup$ @jbuddy it might be better if you undo the changes (you can go back to a previous version) and post an entirely new question. $\endgroup$ Nov 3, 2021 at 18:24
  • 1
    $\begingroup$ @SextusEmpiricus, done! stats.stackexchange.com/questions/550811/… $\endgroup$
    – jbuddy_13
    Nov 3, 2021 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.