# Testing if a coin is fair

I flip a coin 20 times and get 14 heads. I want to calculate the p-value of the hypothesis that my coin is fair.

What probability should I calculate?

Wikipedia says that I need to calculate the probability to get 14 or more heads in 20 flips.

Why is it 14 "or more"? Why not 14 or less?

• Haven't read the Wikipedia entry (which one?), but: this problem is not fully specified. In the absence of more information I would say you should calculate P(H>=14 OR H<=6), i.e. a two-tailed test (the probability, under the null hypothesis (H=10) that the value deviates by as much or more than observed in either direction from the expected value under the null). – Ben Bolker Dec 16 '10 at 16:31
• I suggest you have a look at "L. D. Brown, T. T. Cai and A. DasGupta, Interval Estimation for a Binomial Proportion, Statistical Science, Vol. 16, pp. 101-117, 2001". It includes a very nice discussion of interval estimation for the Binomial distribution. – emakalic Dec 16 '10 at 23:20

Let us then adopt the best symmetric test for a fair coin. This means we want the critical region to include $20-i$ heads whenever it includes $i$ heads (that is, $20-i$ tails). This treats heads and tails on an equal footing. It is only in the context of a particular test (like this one) that a p-value has any meaning. The p-value corresponding to an outcome of 14 is, by definition, the smallest significance of any such test that includes 14 in its critical region. By symmetry it must include 20 - 14 = 6 and by the Neyman-Pearson lemma it must include all values larger than 14 and all values less than 6. The chance of this under the null is 11.53%. This chance increases uniformly as the chance of heads deviates more and more from 1/2 in either direction.