1
$\begingroup$

Suppose we have an extremely large collection of red balls and green balls. If we let $R$ and $G$ be the events of drawing a red ball and drawing a green ball, respectively, and if we let $\Pr(R)=p$ then $\Pr(G)=1-p$. We also assume that the vast majority of the balls in our collection are green, implying $p<<1-p$.

We would like to estimate $p$ using the following experiment: sample from the collection 3 times by selecting 20 balls each time. Then starting with the uninformed prior distribution of $p$ as Beta$(1,1)$ we update the prior by Bayesian updating after each sample and compute the expected value of $p$ after completing the 3 samples.

So suppose we sample and get 20 green balls for each of the 3 samples. The final update of the prior distribution of $p$ is Beta$(1,61)$ and the expected value of $p$ is $E(p)=\frac{1}{62}\approx 0.01613$.

This seems to be a very high estimate and I think the problem is that I am starting the updating with an uninformed prior despite the fact that I know $p<< 1-p$. Is there a way to justify a more informed prior in order to get a stronger conclusion? For example, if I believe that $p\le 0.01$ start with the informed prior Beta$(2,100)$, my update for $p$ will be distributed as Beta$(2,160)$ and $E(p) = \frac{1}{81}>0.01$! That makes no sense to me. Any help would be appreciated.

$\endgroup$

1 Answer 1

4
$\begingroup$

Seeing 0/60 isn't strong evidence for $p\leq 0.01$, since seeing 0/60 is pretty much what you expect for $p\leq 1/60$. This means your posterior won't have high weight on $p\leq 0.01$ unless your prior does

If you are confident a priori that $p\leq 0.01$ you want the prior probability of that to be high. A Beta(2,100) prior still only has 26% probability of $p\leq 0.01$; the Beta(2,160) prior has about 48% probability, which seems plausible. Note that the prior and posterior are very skewed, so although the posterior median is close to 0.01, the posterior mean is higher.

If you had a Beta(2,200) prior, the prior mean would be 0.01 and the prior median a bit lower. The posterior mean would be about .0075, the posterior median about 0.0065, and the posterior probability of $p\leq 0.01$ about 75%

So, there's two things going on: the data don't provide much evidence that $p$ is very small rather that just small, and your prior is much weaker than what you describe as your actual beliefs.

$\endgroup$
1
  • $\begingroup$ Thank you! Your comments make perfect sense. If I solve $(1-p)^{60}=0.5$ for $p$, then that tells me I have a $50\%$ chance that $p\approx 0.011486$ based on the test results, so starting with a prior distribution of Beta$(2,160)$ is reasonable. Then the posterior is distributed as Beta$(2,220)$ so that $E(p)=\frac{1}{111}$. $\endgroup$ Commented Oct 24, 2020 at 5:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.