0
$\begingroup$

Setting

I have a question on the "Cookie Problem Revisited" exercise from Allen Downey's Think Bayes 2e. The Bayes theorem is defined as:

$$ P(H | E) = \frac{P(H) \ P(E | H)}{P(E)} $$

where E is the evidence and H is the hypothesis.

Let's consider the following examples: two bowls contain different cookies.

  • Bowl 1 contains 30 vanilla cookies and 10 chocolate cookies.
  • Bowl 2 contains 20 vanilla cookies and 20 chocolate cookies.

We randomly chose a bowl and draw a vanilla cookie. What is the probability that it came from Bowl 1?

$P(H=\text{Bowl1}) = 0.5$ (we randomly select the bowl)

$P(E=\text{vanilla} \ | \ H=\text{Bowl1}) = 30/(30+10) = 3/4 $

$P(E=\text{vanilla}) = (30 + 20)/(30 + 20 + 10 + 20) = 5/8$ (vanilla cookies over total cookies)

Then, the posterior $P(H=\text{Bowl1} \ | \ E=\text{vanilla}) = 0.6 $.

Question

Now we place the vanilla cookie back into the bowl and draw a second cookie. We get vanilla again. We can use the posterior we have just calculated as the new prior. The posterior is now: $P(E=\text{vanilla} \ | \ H=\text{Bowl1}) * P(H=\text{previous posterior})$ as calculated earlier. However, the probability of the evidence must be different from what we computed earlier (vanilla cookies over total cookies), because otherwise the posterior for Bowl 1 and Bowl 2 would not sum up to 1 (I haven't expressed the posterior for Bowl 2 in this text to keep it simple).

Why isn't the probability of the evidence equal to the ratio of vanilla cookies over total cookies when we draw multiple times? And how can we calculate it in this case?

EDIT: Additional info for clarity

Improve this question
$\endgroup$
3
  • $\begingroup$ I assume that the vanilla cookie is not placed again into the bowl? $\endgroup$
    – Fiodor1234
    Jul 15, 2021 at 16:49
  • $\begingroup$ @Fiodor1234 I forgot that detail, sorry. The vanilla cookie is placed again into the bowl. $\endgroup$
    – maurock
    Jul 15, 2021 at 16:51
  • $\begingroup$ Then the evidence function doesn't change it will always be $5/8$ as you calculated it. Because the evidence will be $p(E_{2}|E_{1})$ what did you draw on the first and second trial, but because you replace it it will be the same I belive $\endgroup$
    – Fiodor1234
    Jul 15, 2021 at 16:53

1 Answer 1

Reset to default
0
$\begingroup$

Suppose that you have information from two draws let's say $E_{1}=vanilla$ and $E_{2}=vanilla$, and you want to calculate the probability of coming from the first bowl, then you will have

$$p(H=B_{1}|E_{1}=vanilla,E_{2}=vanilla)=\frac{p(E_{2}=vanilla|H=B_{1},E_{1}=vanilla)p(H=B_{1}|E_{1}=vanilla)}{p(E_{2}=vanilla|E_{1}=vanilla)}$$

$E_{2}$ is independent of $E_{1}$ conditional on $H=B_{1}$, hence you have

$$p(H=B_{1}|E_{1}=vanilla,E_{2}=vanilla)=\frac{p(E_{2}=vanilla|H=B_{1})p(H=B_{1}|E_{1}=vanilla)}{p(E_{2}=vanilla|E_{1}=vanilla)}$$

where you can calculate the $p(E_{2}=vanilla|E_{1}=vanilla)$ with the law of total probability which eventually will be equal to $5/8$ because you have replacement

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.