# Number of coin tosses needed to establish bias

Say, you toss a coin and see tails, and repeat the toss. How many consecutive tails would you need to establish that a coin is biased ($$P_\textrm{heads} < 0.5$$)

Null: $$P_\textrm{heads} = p = 0.5.$$

So need $$(1-p)^n < 0.05$$ to establish bias at $$95\%$$ confidence one sided \begin{align} \implies 0.5^n &< 0.05\\\implies n &> \frac{\log(0.05)} {\log(0.5)} \\\implies n &\geq 5. \end{align} So, it seems like we only need $$5$$ tails to conclude that coin is biased? That seems very low. Is there something I'm missing?

• The plot at the end of my answer at stats.stackexchange.com/a/620591/919 is relevant: it graphs the chance of obtaining a run of $k$ tails in $100$ tosses (and gives a procedure to do this calculation for any number of tosses). The solution is amenable to asymptotic analysis, showing the correct expression to use in place of "$(1-p)^k$" (which does not account for the number of tosses and therefore cannot be correct).
– whuber
Jul 17, 2023 at 18:05

Comment.

Yes. However, you'd have to be really careful how you used this criterion of getting five Tails in a row to declare a coin as biased. (More-straightforward tests look at the overall balance between Heads and Tails.)

In the experiment below, I simulated 1000 tosses of a fair coin. The longest run of Tails in those 1000 tosses was of length 8. The R procedure rle counts runs of Heads (1's) and Tailz (0's).

set.seed(1234)
x = rbinom(1000, 1, .6)
rle(x)
Run Length Encoding
lengths: int [1:511] 1 5 2 1 1 1 2 1 1 1 ...
values : int [1:511] 1 0 1 0 1 0 1 0 1 0 ...
max(rle(x)$$len[rle(x)$$val==0])
[1] 8


This same sequence of 1000 tosses happened to have a run of 10 Heads.

max(rle(x)$$len[rle(x)$$val==1])
[1] 10

• Good point, however, I was mainly implying that you tossed n times, and only observed tails- not that you observed n consecutive tails in a longer chain of tosses. Sorry for the confusion. May 23, 2021 at 1:59
• I guessed that's want you intended. But it might be a slippery slope. What if you got a run of six Tails after seeing just one Head? May 23, 2021 at 2:10

Rather than counting tails, I would prefer to calculate a minimum sample size to conclude whether the coin may be biased. Here is one way this can be done, using Chebyshev's inequality. The following formula uses this inequality to get a sample size needed to estimate a mean that is within $$\epsilon$$ of the true one with probability $$1-\delta$$, given that we have an upper bound on the variance $$\sigma^2$$: $$ceil(\frac{\sigma^2}{\delta\epsilon^2}).$$

In this case, the variance $$\sigma^2$$ is bounded above by 1/4, which is the maximum variance a Bernoulli random variable can have, namely the variance for a fair coin ($$(1/2)(1-1/2)$$).

For example, we choose $$\epsilon=1/10$$ and $$\delta=1/20$$ (so we have a 95% confidence interval that the probability would be between 4/10 and 6/10 if the coin were fair), then we should flip the coin at least—$$t=ceil(\frac{\frac{1}{4}}{\frac{1}{20}*(\frac{1}{10})^2}) = 500$$ times, count the number of heads, and divide by the number of flips to get the estimated probability of heads, which will be within 1/10 of the true probability at a 95% or greater chance.

Another way is Hoeffding's inequality. Using the same notation as before, the sample size for random variables in [0, 1] (including Bernoulli random variables) is— $$ceil(\frac{\ln(2/\delta)}{2\epsilon^2}),$$ so that the same example becomes— $$ceil(\frac{\ln(2/\frac{1}{20})}{2(\frac{1}{10})^2}) = 185.$$

Actually the exact minimum sample size for $$\epsilon=1/10$$ and $$\delta=1/20$$ is 101. Chen (2011) provided a computational method for computing the minimum sample size as well as a table for common $$\epsilon$$ and $$\delta$$ combinations.
• A large enough $N$ would allow one to detect a trivial bias. It would be perhaps more informative to decide on the minimum level $b$ of bias that matters and compute a Bayesian posterior probability that $P_{heads} > b$. Jul 17, 2023 at 16:29