Relationship between Binomial distribution and the Beta distribution [duplicate]

Question

I have been investigating the details of the Beta distribution and the Binomial distribution and have 2 questions to ask, but first a slight preamble to explain the background to my questions. In the following, I am assuming that the normal parameters for the Beta and Binomial distribution ($\alpha, \beta, n, k$) are positive integers greater than zero, and $x$ is the target probability that we are looking for.

So, in my investigation, I started with the equation for the Beta distribution in the form $Beta(x|\alpha,\beta) = \frac {\Gamma (\alpha +\beta )}{\Gamma (\alpha) \Gamma(\beta )}x^{\alpha-1}(1-x)^{\beta-1} $ and the Binomial distribution in the form $Bin(x|n,k) = {\binom {n}{k}}x^{k}(1-x)^{n-k}$. Then using the fact that $\Gamma(x+1) \equiv x!$, I re-worked the equations for the Beta distribution and the Binomial distribution and came to the following set of four relationships between the two. They are all essentially the same, just expressed in different formats, i.e. going from Beta to Binomial, or Binomial to Beta, and interchanging their normal parameters. To get to the four relationships, the following intermediate relations were derived from the equivalence of the Beta and Binomial distributions:

$\alpha = k + 1; \beta = n - k + 1; n = \alpha + \beta - 2; k = \alpha - 1$

The four relationships are:

1. $Beta(x|\alpha,\beta) = Bin(x|\alpha+\beta-2, \alpha-1)*(\beta+\alpha-1)$
2. $Beta(x|\alpha, \beta) = Bin(x|n,k) * (n+1)$ where $ n = \alpha + \beta - 2$ and $ k = \alpha - 1$
3. $Bin(x|n,k) = \frac{Beta(x|k+1, n-k+1)}{n+1}$
4. $Bin(x|n,k) = \frac{Beta(x|\alpha, \beta)}{\alpha+\beta-1}$ where $\alpha = k + 1$ and $\beta = n - k + 1$

I tested these numerically and they agree, at least for the number set I used. As the above description is quite condensed, and I am happy to elaborate on how I reached these conclusions if required.

So my questions are: 1. Is my analysis correct, and the conclusion that the Binomial and Beta distributions are closely linked as shown above? 2. If it is correct, can anyone point me to any references for more detail on these relationships?

Answer 1

One common usage of the close relationship between the beta and binomial distributions arises in Bayesian statistics. Suppose I want an interval estimate of binomial $p$ based on $x$ successes in $n$ independent trials.

If we have advance information about $p$ we might express it in terms of a prior distribution on $p.$ (In Bayesian statistics parameters can be random variables with probability distributions.)

Maybe $p$ is the unknown prevalence of a disease in a certain population. Perhaps we have reason to believe that $p \approx 0.1$ and we are pretty sure that $p < 0.3.$ Then we might use prior distribution $\mathsf{Beta}(\alpha = 1, \beta = 9)$ with density function $$\frac{\Gamma(10)}{\Gamma(1)\Gamma(9)}p^{1-1}(1-p)^{9-1} = 9(1-p)^8,$$ so that $E(p) = \frac{\alpha}{\alpha+\beta} = 0.1$ and $P(p < .03) \approx 0.96.$ (Computation in R below.)

pbeta(.3, 1, 9)
# [1] 0.9596464

If reliable testing on $n=1000$ randomly chosen members of the population show that $x = 23$ have the disease, then we have the binomial likelihood function $f(x|p) = {n\choose x}p^x(1-p)^{n-x}.$ It is common practice to express likelihood functions without the "norming" constant that makes the corresponding density function sum (or integrate) to unity, so we write $f(x|p) \propto p^{23}(1-p)^{927},$ where the symbol $\propto$ is read "proportional to."

Then by a version of Bayes' Theorem, we say that the posterior distribution is found by taking the product of the prior distribution and the likelihood function for the data:

$$f(p|x) \propto f(x) \times f(x|p) = p^{\alpha-1}(1-p)^{\beta-1} \times p^{x}(1-p)^{n-x}\\ = (1-p)^{9-1}\times p^{23}(1-p)^{927} = p^{23}(1-p)^{935},$$ where we easily recognize the final member of this relationship to be the 'kernel' (density without constant) of the posterior distribution $\mathsf{Beta}(24, 936).$

Note: The close relationship you noted between the binomial PDF and the beta density function made it possible to recognize the posterior distribution without further computation. We say that a beta prior is 'conjugate' to the binomial likelihood.

We say that the Bayesian posterior mean $E(p|x) = \frac{24}{24+936} = 0.026$ is a point estimate of disease prevalence and we cut probability 0.025 from each tail of $\mathsf{Beta}(24, 936)$ to obtain the 95% Bayesian posterior probability interval ${0.033. 0.036)$ is an interval estimate based on prior information and testing data.

qbeta(c(.925,.975), 24, 935)
[1] 0.03260073 0.03580752

If we have no useful prior information, it is customary to use a noninformative prior distribution, such as $\mathsf{Beta}(.5,.5).$ Then the posterior distribution would be $\mathsf{Beta}(23.5, 927.5),$ which would have given the 95% Bayesian interval estimate $(0.016, 0.035),$ fundamentally based on data alone. It is possible for the prior distribution to have a major influence on Bayesian point and interval estimates. (Sometimes that may be the whole point of using Bayesian inference.) However, in this example, the influence of our prior distribution was relatively small.

qbeta(c(.025,.975), 23.5, 927.5)
[1] 0.01582941 0.03548037

For comparison, a frequentist Wald 95% confidence interval based on data is $(0.014, 0.032),$ using the point estimate $\hat p = 0.023.$

p.est = 23/1000
p.est + c(-1.96, 1.96)*sqrt(p.est*(1-p.est)/1000)
[1] 0.0137089 0.0322911