How to assess whether a coin tossed 900 times and comes up heads 490 times is biased?

Question

A coin is tossed 900 times and heads appeared 490 times. Does the result support the hypothesis that the coin is unbiased?

Answer 1

If the coin is unbiased then the probability of 'heads' is $\frac{1}{2}$. Therefore, the number of heads thrown in 900 tries, $X$, has a ${\rm Binomial}(900,\frac{1}{2})$ distribution under the null hypothesis of a fair coin. So, the $p$-value - the probability of seeing a result this extreme or more extreme given that the coin is far, is

$$ P( X \geq 490 ) $$

If you seek the 2-sided $p$-value, that would be

$$ 1 - P(410 < X < 490 ) $$

I'll leave it to you to describe why that is the case.

We know that the mass function for $ Y \sim {\rm Binomial}(n,p)$, is

$$ P(Y = y) = \binom{n}{y} p^y (1-p)^{n-y} $$

I'll leave it to you to calculate $p$-value you seek.

Note: The sample size here is sufficiently large that you could use the normal approximation to the binomial distribution. I've detailed above how to calculate the exact $p$-value.

Answer 2

Here the natural null-hypothesis $H_0$ is that the coin is unbiased, that is, that the probability $p$ of a head is equal to $1/2$. The most reasonable alternate hypothesis $H_1$ is that $p\ne 1/2$, though one could make a case for the one-sided alternate hypothesis $p>1/2$.

We need to choose the significance level of the test. That's up to you. Two traditional numbers are $5$% and $1$%.

Suppose that the null hypothesis holds. Then the number of heads has *binomial distribution with mean $(900)(1/2)=450$, and standard deviation $\sqrt{(900)(1/2)(1/2)}=15$.

The probability that in tossing a fair coin the number of heads differs from $450$ by $40$ or more (in either direction) is, by symmetry, $$2\sum_{k=490}^{900} \binom{900}{k}\left(\frac{1}{2}\right)^{900}.$$ This is not practical to compute by hand, but Wolfram Alpha gives an answer of roughly $0.008419$.

Thus, if the coin was unbiased, then a number of heads that differs from $450$ by $40$ or more would be pretty unlikely. It would have probability less than $1$%. so at the $1$% significance level, we reject the null hypothesis.

We can also use the normal approximation to the binomial to estimate the probability that the number of heads is $\ge 490$ or $\le 410$ under the null hypothesis $p=1/2$. Our normal has mean $450$ and variance $15$ is $\ge 490$ with probability the probability that a standard normal is $\ge 40/15$. From tables for the normal, this is about $0.0039$. Double to take the left tail into account. We get about $0.0078$, fairly close to the value given by Wolfram Alpha, and under $1$\%. So if we use $1$\% as our level of significance, again we reject the null hypothesis $H_0$.

Comments: $1$. In the normal approximation to the binomial, we get a better approximation to the probability that the binomial is $\ge 490$ by calculating the probability that the normal is $\ge 489.5$. If you want to look it up, this is the continuity correction. If we use the normal approximation with continuity correction, we find that the probability of $490$ or more or $410$ or fewer heads is about $0.008468$, quite close to the "exact" answer provided by Wolfram Alpha. Thus we can find a very accurate estimate by, as in the bad old days, using tables of the standard normal and doing the arithmetic "by hand."

$2$. Suppose that we use the somewhat less natural alternate hypothesis $p>1/2$. If $p=1/2$, the probability of $490$ or more is about $0.00421$. Thus again at the $1$% significance level, we would reject the null hypothesis, indeed we would reject it even if we were using significance level $0.005$.

Setting a significance level is always necessary, for it is possible for a fair coin to yield say $550$ or more heads in $900$ tosses, just ridiculously unlikely.

Answer 3

Your question could be addressed in a few different ways.

The traditional test of hypothesis is designed to rule out possibilities, not necessarily prove them. In this case we can use $H_0: p=0.5$ as the null hypothesis and see if the data (the 490 out of 900 heads) can be used to reject this null hypothesis by computing a p-value. If the p-value is less than $\alpha$ then we reject the null, but a p-value $>\alpha$ does not mean that we can say the data supports the null, just that it is consistant with the assumption that the null is true, but in truth the null could be false, just the truth is a value of $p$ very close to $0.5$.

The "equivalence" approach would be to define unbiased not as $p=0.5$ but rather choose a small region around 0.5 to consider as unbiased $ 0.5-\epsilon < p < 0.5+\epsilon$. Then if the confidence interval on the true proportion lies fully within the equivalence interval of "unbiased" then the data would support the hypothesis of "unbiasedness".

Another approach would be to use a Bayesian approach where we start with a prior distribution on the true proportion $p$ including a point mass at 0.5 and the rest of the probabilit spread accross possible values. Then combine that with the data to get a posterior. If the posterioun probabilit of $p=0.5$ is high enough then that would support the claim of being unbiased.

Answer 4

The example from the Wikipedia page on Bayes Factor seems quite relevant to the question. If we have two models, M1 where the coin is exactly unbiased (q=0.5), and M2 where the probability of a head is unknown, so we use a flat prior distribution on 1. We then compute the bayes factor

$K = \frac{p(x=490|M_0)}{p(x=490|M_1)}$

where

$p(x=490|M1) = \mathrm{nchoosek}(900,490)\frac12^{900} = 7.5896\times10^{-4}$

and

$p(x=490|M2) = \int_0^1 \mathrm{nchoosek}(900,490)q^{490}(1-q)^{410}dq = \frac{1}{901}$

Gives a Bayes factor of $K \approx 1.4624$, which according to the usual scale of interpretation is "barely worth mentioning".

Note however (i) the Bayes factor has a built in occam penalty that favours simple models, and M1 is a lot simpler as it has no nuisance parameters, where as M2 does; (ii) a flat prior on $q$ is not physically reasonable, in practice a biased coin is going to be close to fair unless the coin is obviously assymetrical; (iii) it has been a long day and I could easily have made a mistake some(any)where in the analysis from assumptions to calculations.

Note that the coin is biased if it is a physical object as its assymetry means that it won't be exactly as likely to come down heads as tails.

Answer 5

To clarify the Bayesian approach:

You start by knowing nothing, except that P(Heads) is in [0,1]. So start with a maximum entropy prior -> uniform(0,1). This can be represented as a beta distribution -> beta(1,1).

Each time you flip the coin do a Bayesian up-date of the coin's P(Heads) by multiplying each point in the distribution by it's likelihood (multiply by x if you roll heads, multiply by (1-x) if you get tails), and re-normalize the total probability to 1. This is what the beta distribution does, so if the first roll is heads you'll have beta(2,1). In your case you have beta(490,510).

From there I would calculate the 95% probability interval, and if 0.5 isn't in that interval, I would start to get suspicious.

The first time I went through this exercise I was really surprised at how long it took to converge... I started because someone said "if you flip a coin 100 times, you know P(Heads) to +/- 1%" this turns out to be totally wrong, you need magnitudes more than 100 flips.

Answer 6

And an R illustration:

Not bothering to approximate by the normal, we can look at a random variable distributed binomial with n=900 and p=0.5 under the null hypothesis (i.e. if the coin were unbiased then p=probability of heads(or tails) = 0.5).

If we would like to test the alternative that Ha: p<>0.5 at alpha 0.05 we can look at the tails of the distribution under the null as follows and see that 490 falls outside the interval {421, 479} and thus we reject Ho.

n<-900
p<-0.5
qbinom(c(0.025,0.975),size=n,prob=p)
# 421 479