# Bayesian Posterior distribution for binomial distribution with uniform prior

Suppose we have two independent binomial distribution given p, i.e. $$X_1|p \sim Bin(n_1, p)$$, $$X_2|p \sim Bin(n_2, p)$$. We also know the prior distribution for p is $$p \sim U(0,1)$$. Now I would like to find the posterior distribution P|(X1, X2). Here is my calculation.

Our prior density for p is \begin{align*} \pi(p) = I(1 > p > 0) \end{align*}

The likelihood for $$(X_1, X_2)$$ given p is \begin{align*} f(x_1, x_2|p) &= P(X_1=x_1, X_2=x_2|p) \\&= P(X_1=x_1|p)P(X_2=x_2|p) \\&= {{n_1}\choose{x_1}}p^{x_1}(1-p)^{n_1-x_1} {{n_2}\choose{x_2}}p^{x_2}(1-p)^{n_2-x_2} \end{align*}

Thus, the posterior distribution is \begin{align*} \pi(p|x_1,x_2) &\propto f(x_1, x_2|p)\pi(p) \\&\propto p^{x_1+x_2}(1-p)^{n_1+n_2-x_1-x_2} \end{align*}

Thus, $$P|X_1,X_2 \sim Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$$

Finally, the posterior mean, i.e. the Bayes estimator for p is the mean of $$Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$$, \begin{align*} E(P|X_1,X_2) = \frac{X_1+X_2+1}{n_1+n_2+2} \end{align*}

I am just wondering if my calculation is correct. I saw examples from the textbook, there is the example for n iid Bernoulli(p) trial with $$p \sim U(0,1)$$ prior, and there is also the example for $$X|p \sim Bin(n, p)$$ with $$p \sim U(0,1)$$ prior. But I haven't seen an example of my derivation.

Suppose now $$n_1=400$$, $$n_2=600$$, $$X_1=10$$, $$X_2=200$$, then my Bayes estimator for p is $$\frac{211}{1002}$$. While if we only consider $$X_1$$ and $$n_1$$, the Bayes estimator for p is $$\frac{X_1+1}{n_1+2} = \frac{11}{402}$$, if we only consider $$X_2$$ and $$n_2$$, the Bayes estimator for p is $$\frac{X_2+1}{n_2+2} = \frac{201}{602}$$. The three Bayes estimators are quite different. That's why I doubt if my calculation makes sense.

For my case, $$X_1|p \sim Bin(n_1, p)$$, $$X_2|p \sim Bin(n_2, p)$$, the p in $$Bin(n_1, p)$$ is the same number as the p in $$Bin(n_2, p)$$, they not only share the same $$U(0,1)$$ distribution, but also are identical number, is my interpretation correct? Or they only share the same $$U(0,1)$$ distribution, but are not identical number.

• Your posterior $Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$ looks correct and its mean is indeed $\frac{X_1+X_2+1}{n_1+n_2+2}$. Meanwhile your $n_1=400$, $n_2=600$, $X_1=10$, $X_2=200$ example is extreme in the sense that it is unlikely to occur if $p$ is the same in both cases, so you should not be at all surprised that using one of $X_1$ or $X_2$ would give a very different result compared to using both together Commented Oct 4, 2022 at 1:14
• Indeed, and the "extremity" observed here also is unrelated to the Bayesian route taken. The MLEs (ie, the sample proportions computed in the answer by jbowman below) are also vastly different. Commented Oct 4, 2022 at 9:17

To answer your last question first - the way you have written it, the $$p$$ is the same in the two distributions, so they share the same prior.
As for why the three estimators give different values - look at the sample proportions of $$X_1$$ and $$X_2$$ in your example. For $$1$$, you have 2.5% of the observations for which $$X_1 = 1$$; for $$2$$, you have 33% of the observations for which $$X_2 = 1$$. This is extremely unlikely to happen with $$n_1 = 400$$, $$n_2 = 600$$, and the same probability, to put it mildly. Given the widely different results of your two samples, your sample sizes in the hundreds, and the uninformative prior on $$p$$, it's natural that the estimators will be very different as well.
• For the last question, I mean if p are also the identical number. For example, suppose we generate a random number 0.3 from U(0, 1), then p for $X_1$ and $X_2$ are all 0.3. Or we generate two random number 0.3, 0.5 from U(0,1), p for $X_1$ is 0.3, p for $X_2$ is 0.5. Thanks for your help! Commented Oct 4, 2022 at 1:21
• I would interpret it as being that $p$ is the identical number in the two distributions; the way the formulae are written, that would be the correct interpretation at any rate. Commented Oct 4, 2022 at 1:45