I've been working on a problem about coins and Bayes' theorem, but I'm getting some counterintuitive results. Here's the problem and what I've tried.
Imagine you have a coin, and you have no reasons to assume it's biased, so the prior is $P(heads) = 0.5$. Now imagine you've observed the data $D = \{heads:120, tails:100\}$, and you want to compute the expected heads probability after having observed this data. If we apply the Bayes rule we get
$$ P(head | D) = \frac{P(D|head) P(head)}{P(D)} $$
where
- The prior is $P(head) = 0.5$, since we're assuming it's an unbiased coin. Therefore, $P(tail) = 0.5$ as well.
- The likelihood is $P(D|head) = Binomial(220, 100, 0.5) = \binom{220}{120}0.5^{120}0.5^{100}$
- $P(D) = P(D|head)P(head) + P(D|tail)P(tail)$, where $P(D|tail) = \binom{220}{100}0.5^{100}0.5^{120}$
If we plug everything in the Bayes rule we get
$$ P(head|D) =\frac{\binom{220}{120}0.5^{120}0.5^{100}0.5}{\binom{220}{120}0.5^{120}0.5^{100}0.5 + \binom{220}{100}0.5^{100}0.5^{120}0.5} = \frac{\binom{220}{120}}{\binom{220}{120} + \binom{220}{100}} = \frac{1}{2} $$
That seems counterintuitive to me. I would expect a posterior probability higher than $0.5$, something around $120/220\approx 0.545$.
Notice also that if we change the observed data to $D=\{heads: 12000, tails: 10000\}$ the results are the same even if we have much more information.
How is this possible? Is there some error in my numbers? Did I understand something wrong?
Thanks for your help!