I know that the beta distribution is conjugate to the binomial. But what is the conjugate prior of the beta? Thank you.
$\begingroup$
$\endgroup$
2
asked Aug 15, 2013 at 0:33
-
1$\begingroup$ If one works with one parameter fixed, say Beta$(\theta,1)$, then a conjugate prior for $\theta$ is a Gamma distribution. $\endgroup$– StubbornAtomApr 14, 2020 at 18:45
-
1$\begingroup$ Also see the question Random number generation for conjugate distribution of beta distribution. $\endgroup$– MankkaNov 5, 2020 at 14:00
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
9
It seems that you already gave up on conjugacy. Just for the record, one thing that I've seen people doing (but don't remember exactly where, sorry) is a reparameterization like this. If $X_1,\dots,X_n$ are conditionally iid, given $\alpha,\beta$, such that $X_i\mid\alpha,\beta\sim\mathrm{Beta}(\alpha,\beta)$, remember that $$ \mathbb{E}[X_i\mid\alpha,\beta]=\frac{\alpha}{\alpha+\beta} =: \mu $$ and $$ \mathbb{Var}[X_i\mid\alpha,\beta] = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} =: \sigma^2 \, . $$ Hence, you may reparameterize the likelihood in terms of $\mu$ and $\sigma^2$ and use as a prior $$ \sigma^2\mid\mu \sim \mathrm{U}[0,\mu(1-\mu)] \qquad \qquad \mu\sim\mathrm{U}[0,1] \, . $$ Now you're ready to compute the posterior and explore it by your favorite computational method.
-
6$\begingroup$ No, not MCMC this thing! Quadrature this thing! only 2 parameters - quadrature is the "gold standard" for small dimensional posteriors, both for time and accuracy. $\endgroup$ Aug 18, 2013 at 22:47
-
5$\begingroup$ Another option is to regard $\psi = \alpha + \beta$ as a measure of precision, and again use $\mu = \frac{\alpha}{\alpha + \beta}$ as an the mean. This is done all the time with Dirichlet processes, and the beta distribution is a special case. So maybe toss a gamma or log-normal prior on $\psi$ and uniform on $\mu$. $\endgroup$– guyAug 18, 2013 at 23:34
-
3
-
3
-
$\begingroup$ Hi @Zen i'm dealing with this problem right now, but i'm new at Bayesian and im not sure if im understanding the idea. I figured out that you're proposing to find $\int_{0}^{1}{\frac{1}{\mu(1-\mu}}d\mu$ and then use reparametrization, but of course that Was not the idea. Can you please help me to understand? $\endgroup$ Mar 12, 2017 at 15:54
$\begingroup$
$\endgroup$
9
Yes, it has a conjugate prior in the exponential family. Consider the three parameter family $$ \pi(\alpha, \beta \mid a, b, p) \propto \left\{\frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)}\right\}^p \exp\left(a\alpha + b\beta \right). $$ For some values of $(a, b, p)$, this is integrable, although I haven't quite figured out which ones (I believe $p \ge 0$ and $a < 0, b < 0$ should work — $p = 0$ corresponds to independent exponential distributions so that definitely works; moreover, the conjugate update involves incrementing $p$ so this suggests that $p > 0$ works as well).
The problem, and at least part of the reason no one uses it, is that $$ \int_0^\infty \int_0^\infty \left\{\frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)}\right\}^p \exp\left(a\alpha + b\beta \right) = ? $$ i.e. the normalizing constant doesn't have a closed form.
-
$\begingroup$ Ah. That is problematic. I was going to look for an uninformative version of the conjugate prior anyway, so looks like I might as well start with uniform priors over the two parameters. Thanks. $\endgroup$ Aug 15, 2013 at 20:42
-
1$\begingroup$ You don't need to normalize it if you're just comparing likelihoods… $\endgroup$– Neil GAug 17, 2013 at 23:26
-
$\begingroup$ I think you might be missing the action of $p$ in your $\exp$ term as well. It should probably be $pa\alpha$, etc. $\endgroup$– Neil GAug 17, 2013 at 23:38
-
$\begingroup$ @NeilG $p$ is in the $\exp$, you just have to express things in terms of $\log \Gamma(\cdots)$ instead of $\Gamma(\cdots)$. Doing $pa\alpha$ is just a reparmetrization, it changes nothing. Not sure what you mean "just comparing likelihoods". You can't implement a Gibbs sampler with this prior without using something like Metropolis, which kills the advantage of conditional conjugacy, the normalizing constant depends on $a$ and $b$ which kills putting a prior on them or estimating them by likelihood methods, etc... $\endgroup$– guyAug 17, 2013 at 23:44
-
2$\begingroup$ @NeilG integral is over $\alpha$ and $\beta$ since those are the random variables. $\endgroup$– guyAug 18, 2013 at 1:59
$\begingroup$
$\endgroup$
1
In theory there should be a conjugate prior for the beta distribution. This is because
- the beta distribution is one of the exponential family distributions, and
- in theory it should be possible to derive a prior. See, e.g., wikipedia, D Blei's lecture on exponential families.
However the derivation looks difficult, and to quote A Bouchard-Cote's Exponential Families and Conjugate Priors
An important observation to make is that this recipe does not always yields a conjugate prior that is computationally tractable.
Consistent with this, there is no prior for the Beta distribution in D Fink's A Compendium of Conjugate Priors.
-
4$\begingroup$ The derivation is not difficult — See my answer: mathoverflow.net/questions/63496/… $\endgroup$– Neil GAug 17, 2013 at 23:27
$\begingroup$
$\endgroup$
2
Robert and Casella (RC) happen to describe the family of conjugate priors of the beta distribution in Example 3.6 (p 71 - 75) of their book, Introducing Monte Carlo Methods in R, Springer, 2010. However, they quote the result without citing a source.
Added in response to gung's request for details. RC state that for distribution $B(\alpha, \beta)$, the conjugate prior is "... of the form
$$ \pi(\alpha,\beta) \propto \Big\{ \frac{\Gamma(\alpha+\beta)} {\Gamma(\alpha)\Gamma(\beta)} \Big\} ^{\lambda} x_0^{\alpha} y_0^{\beta} $$
where $\{\lambda, x_0, y_0\}$ are hyperparameters, since the posterior is then equal to
$$ \pi(\alpha,\beta \vert x) \propto \Big\{ \frac{\Gamma(\alpha+\beta)} {\Gamma(\alpha)\Gamma(\beta)} \Big\} ^{\lambda} (xx_0)^{\alpha} ((1-x)y_0)^{\beta}." $$
The remainder of the example concerns importance sampling from $\pi(\alpha,\beta \vert x)$ in order to compute the marginal likelihood of $x$.
-
3$\begingroup$ I don't have Robert's book available but the posterior is $\pi(\alpha, \beta) \propto \left( \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha) \Gamma(\beta)} \right)^{\lambda +1} (x x_0)^{\alpha-1} \left(y_0 (1-x) \right)^{\beta-1}$. Robert also posted on this topic here mathoverflow.net/questions/20399/… $\endgroup$ Jan 13, 2016 at 20:22
-
2$\begingroup$ I humbly recommend that the original poster updates the post to indicate that the posterior given in the textbook is incorrect, per Fred Schoen's comment (which is easily verified). $\endgroup$– RMurphyMay 29, 2017 at 21:04
$\begingroup$
$\endgroup$
4
I do not believe there is a "standard" (i.e., exponential family) distribution that is the conjugate prior for the beta distribution. However, if one does exist it would have to be a bivariate distribution.
-
$\begingroup$ I have no idea about this question, but I did find this handy conjugate prior map that seems to support your answer: johndcook.com/conjugate_prior_diagram.html $\endgroup$ Aug 15, 2013 at 3:10
-
1$\begingroup$ The conjugate prior is in the exponential family and has three parameters — not two. $\endgroup$– Neil GAug 17, 2013 at 23:25
-
1$\begingroup$ @Neil, you are definitely right. I guess I should have said it would have to have at least two parameters. $\endgroup$– user25658Aug 18, 2013 at 18:28
-
1$\begingroup$ -1: this answer is clearly wrong in the claim that "conjugate prior does not exist in the exponential family", as is demonstrated in the answer above... $\endgroup$ Apr 24, 2018 at 8:34