1
$\begingroup$

Suppose we have two independent binomial distribution given p, i.e. $X_1|p \sim Bin(n_1, p)$, $X_2|p \sim Bin(n_2, p)$. We also know the prior distribution for p is $p \sim U(0,1)$. Now I would like to find the posterior distribution P|(X1, X2). Here is my calculation.

Our prior density for p is \begin{align*} \pi(p) = I(1 > p > 0) \end{align*}

The likelihood for $(X_1, X_2)$ given p is \begin{align*} f(x_1, x_2|p) &= P(X_1=x_1, X_2=x_2|p) \\&= P(X_1=x_1|p)P(X_2=x_2|p) \\&= {{n_1}\choose{x_1}}p^{x_1}(1-p)^{n_1-x_1} {{n_2}\choose{x_2}}p^{x_2}(1-p)^{n_2-x_2} \end{align*}

Thus, the posterior distribution is \begin{align*} \pi(p|x_1,x_2) &\propto f(x_1, x_2|p)\pi(p) \\&\propto p^{x_1+x_2}(1-p)^{n_1+n_2-x_1-x_2} \end{align*}

Thus, $P|X_1,X_2 \sim Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$

Finally, the posterior mean, i.e. the Bayes estimator for p is the mean of $Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$, \begin{align*} E(P|X_1,X_2) = \frac{X_1+X_2+1}{n_1+n_2+2} \end{align*}

I am just wondering if my calculation is correct. I saw examples from the textbook, there is the example for n iid Bernoulli(p) trial with $p \sim U(0,1)$ prior, and there is also the example for $X|p \sim Bin(n, p)$ with $p \sim U(0,1)$ prior. But I haven't seen an example of my derivation.

Suppose now $n_1=400$, $n_2=600$, $X_1=10$, $X_2=200$, then my Bayes estimator for p is $\frac{211}{1002}$. While if we only consider $X_1$ and $n_1$, the Bayes estimator for p is $\frac{X_1+1}{n_1+2} = \frac{11}{402}$, if we only consider $X_2$ and $n_2$, the Bayes estimator for p is $\frac{X_2+1}{n_2+2} = \frac{201}{602}$. The three Bayes estimators are quite different. That's why I doubt if my calculation makes sense.

For my case, $X_1|p \sim Bin(n_1, p)$, $X_2|p \sim Bin(n_2, p)$, the p in $Bin(n_1, p)$ is the same number as the p in $Bin(n_2, p)$, they not only share the same $U(0,1)$ distribution, but also are identical number, is my interpretation correct? Or they only share the same $U(0,1)$ distribution, but are not identical number.

$\endgroup$
2
  • $\begingroup$ Your posterior $Beta(X_1+X_2+1,n_1+n_2-X_1-X_2+1)$ looks correct and its mean is indeed $\frac{X_1+X_2+1}{n_1+n_2+2}$. Meanwhile your $n_1=400$, $n_2=600$, $X_1=10$, $X_2=200$ example is extreme in the sense that it is unlikely to occur if $p$ is the same in both cases, so you should not be at all surprised that using one of $X_1$ or $X_2$ would give a very different result compared to using both together $\endgroup$
    – Henry
    Oct 4, 2022 at 1:14
  • $\begingroup$ Indeed, and the "extremity" observed here also is unrelated to the Bayesian route taken. The MLEs (ie, the sample proportions computed in the answer by jbowman below) are also vastly different. $\endgroup$ Oct 4, 2022 at 9:17

1 Answer 1

1
$\begingroup$

To answer your last question first - the way you have written it, the $p$ is the same in the two distributions, so they share the same prior.

Your calculation is indeed correct; you've found the posterior distribution and posterior mean.

As for why the three estimators give different values - look at the sample proportions of $X_1$ and $X_2$ in your example. For $1$, you have 2.5% of the observations for which $X_1 = 1$; for $2$, you have 33% of the observations for which $X_2 = 1$. This is extremely unlikely to happen with $n_1 = 400$, $n_2 = 600$, and the same probability, to put it mildly. Given the widely different results of your two samples, your sample sizes in the hundreds, and the uninformative prior on $p$, it's natural that the estimators will be very different as well.

$\endgroup$
2
  • $\begingroup$ For the last question, I mean if p are also the identical number. For example, suppose we generate a random number 0.3 from U(0, 1), then p for $X_1$ and $X_2$ are all 0.3. Or we generate two random number 0.3, 0.5 from U(0,1), p for $X_1$ is 0.3, p for $X_2$ is 0.5. Thanks for your help! $\endgroup$
    – Mizzle
    Oct 4, 2022 at 1:21
  • $\begingroup$ I would interpret it as being that $p$ is the identical number in the two distributions; the way the formulae are written, that would be the correct interpretation at any rate. $\endgroup$
    – jbowman
    Oct 4, 2022 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.