Suppose we have these two sequences of 10 independent coin flips:


And suppose we want to test wether the coin is balanced or not (let $p=P(H)$).

Under $p=\frac{1}{2}$ both sequences have the same probability of $$ \bigg(\frac{1}{2}\bigg)^{10}. $$

I don't understand why, even if these two sequences have the same probability under $p=0.5$, the second one seems more against the hypothesis $p=0.5$ than the first one.

Aside from the Binomial likelihood where combinatorics comes into play to count the number of $H$, why will the these two equally likely sequences lead to different inference?

  • $\begingroup$ Just to clarify, you don't think that getting 10 out of 10 heads is in some way evidence against the coin being fair? $\endgroup$ – jbowman Nov 11 '19 at 15:53
  • $\begingroup$ @jbowman I do! I just don't understand how, given that under $p=0.5$ they are both equally likely, the second one will be more against $p=0.5$ than the first one. $\endgroup$ – rosas Nov 11 '19 at 16:06
  • $\begingroup$ Which one is more likely under an alternative such as $p=0.9$? $\endgroup$ – jbowman Nov 11 '19 at 16:18

Your observation is correct that under $p=0.5$ every observed sequence of heads and tails has the same probability. This means that the precise sequence of heads and tails is not useful as a test statistic for discriminating between $p=0.5$ and $p\neq 0.5$.

That's why the number of heads $k$ (or the proportion of heads $k/n$) is used as a test statistic. Its distribution under $p=0.5$ is known (binomial distribution with $p=0.5$) and you can compute the acceptance region $[k_1,k_2]=[np-\epsilon, np+\epsilon]$ for $p=0.5$ as $$P(|k-np|\leq\epsilon) \approx 1-\alpha$$ where $\alpha$ is the confidence significance level.

| cite | improve this answer | |
  • 1
    $\begingroup$ Good answer, +1. My complaint is calling $\alpha$ the confidence level. We can define anything to be anything we wish, but $\alpha$ has a pretty well established meaning in statistics of being the complement of the confidence level. $\endgroup$ – Dave Nov 11 '19 at 16:50
  • $\begingroup$ @dave Thanks for pointing this out. I have corrected the term "confidence" to "significance". $\endgroup$ – cdalitz Nov 11 '19 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.