How does the beta prior affect the posterior under a binomial likelihood

I have two questions,

Question 1: How can I show that the posterior distribution is a beta distribution if the likelihood is binomial and the prior is a beta

Question 2: How does choices the prior parameters affect the posterior? Shouldn't they all be the same?

Is it possible to answers these questions in R?

• You can easily use firstbayes to make comparison without having to write codes as in R. Get firstbayes. This might help because installing it might be tricky..youtube.com/watch?v=BcdigiWY054
– user42101
Mar 17 '14 at 21:16

To answer your first question we just need to use Bayes' Theorem to update our binomial likelihood with the beta prior. To better understand how to do this, first observe the following result $$p(\theta|\mathbf{x})=\frac{p(\mathbf{x}|\theta)p(\theta)}{\int_{\Theta}p(\mathbf{x}|\theta)p(\theta)d\theta}\propto p(\mathbf{x}|\theta)p(\theta)$$ where we can make use of the proportionality result since the beta distribution is the conjugate prior for the binomial likelihood.

Now, let $x_i\sim\text{Binomial}(N_i,\theta)$ and $\theta\sim\text{Beta}(\alpha,\beta)$. We can now use Bayes' Theorem to calculate the posterior as follows:

\begin{align} p(\theta|\mathbf{x})&\propto p(\mathbf{x}|\theta)p(\theta)\\ &\\ &\propto \binom{N}{x_i}\theta^{s}(1-\theta)^{N-s}\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\theta^{\alpha-1}(1-\theta)^{\beta-1}\\ &\\ &\propto \theta^{s}(1-\theta)^{N-s}\theta^{\alpha-1}(1-\theta)^{\beta-1}\\ &\\ &\propto\theta^{\alpha+s-1}(1-\theta)^{\beta+N-s-1} \end{align} where $s=\sum_{i=1}^nx_i$ and $N=\sum_{i=1}^nN_i$

Now, we recognize the proportional right hand side of the equation as the kernel of another beta distribution with updated parameters $$\alpha^*=\alpha+\sum_{i=1}^nx_i$$ and $$\beta^*=\beta+\sum_{i=1}^nN_i-\sum_{i=1}^nx_i$$

Now for the second part of your problem, consider the following graphs of the posteriors given differing prior distributions. The above plot is of five different prior distributions: \begin{align*} \text{Prior }1&:\,\,\theta\sim\text{Beta}(.5,.5)\\ \text{Prior }1&:\,\,\theta\sim\text{Beta}(5,1)\\ \text{Prior }1&:\,\,\theta\sim\text{Beta}(1,3)\\ \text{Prior }1&:\,\,\theta\sim\text{Beta}(2,2)\\ \text{Prior }1&:\,\,\theta\sim\text{Beta}(2,5) \end{align*}     Now although the posterior distribution does not seem to be changed much by the choice of prior in this situation this is not always the case. For example, had we sampled from a Binomial distribution (in the code) where $N=2$ we would see that the posterior distribution is drastically changed by the choice of the prior distribution.

Here is the R code used to generate everything:

colors = c("red","blue","green","orange","purple")

n = 10
N = 10
theta = .2

x = rbinom(n,N,theta)
grid = seq(0,2,.01)

alpha = c(.5,5,1,2,2)
beta = c(.5,1,3,2,5)

plot(grid,grid,type="n",xlim=c(0,1),ylim=c(0,4),xlab="",ylab="Prior Density",
main="Prior Distributions", las=1)
for(i in 1:length(alpha)){
prior = dbeta(grid,alpha[i],beta[i])
lines(grid,prior,col=colors[i],lwd=2)
}

legend("topleft", legend=c("Beta(0.5,0.5)", "Beta(5,1)", "Beta(1,3)", "Beta(2,2)", "Beta(2,5)"),
lwd=rep(2,5), col=colors, bty="n", ncol=3)

for(i in 1:length(alpha)){
dev.new()
plot(grid,grid,,type="n",xlim=c(0,1),ylim=c(0,10),xlab="",ylab="Density",xaxs="i",yaxs="i",
main="Prior and Posterior Distribution")

alpha.star = alpha[i] + sum(x)
beta.star = beta[i] + n*N - sum(x)
prior = dbeta(grid,alpha[i],beta[i])
post = dbeta(grid,alpha.star,beta.star)

lines(grid,post,lwd=2)
lines(grid,prior,col=colors[i],lwd=2)
legend("topright",c("Prior","Posterior"),col=c(colors[i],"black"),lwd=2)

}
• (+1) Great answer @user25658. Jan 17 '15 at 14:30
• This is great answer and would be fine it also includes likelihood (Data) graph. Jan 17 '15 at 18:17
• @user25658 If the posterior is a beta distribution shouldn't we be using rbeta instead of rbinom to generate x since theta is a beta distr ? Oct 23 '18 at 11:48