Number of coin tosses needed to establish bias Say, you toss a coin and see tails, and repeat the toss. How many consecutive tails would you need to establish that a coin is biased ($P_\textrm{heads} < 0.5$)
Null: $P_\textrm{heads} = p = 0.5.$
So need $(1-p)^n < 0.05$ to establish bias at $95\%$ confidence one sided
\begin{align}
\implies 0.5^n &< 0.05\\\implies n &> \frac{\log(0.05)} {\log(0.5)}
\\\implies n &\geq 5.
\end{align}
So, it seems like we only need $5$ tails to conclude that coin is biased? That seems very low. Is there something I'm missing?
 A: Comment.
Yes. However, you'd have to be really careful how you used
this criterion of getting five Tails in a row to declare a coin
as biased. (More-straightforward tests look at the overall balance between
Heads and Tails.)
In the experiment below, I simulated 1000 tosses of a fair coin.
The longest run of Tails in those 1000 tosses was of length 8.
The R procedure rle counts runs of Heads (1's) and Tailz (0's).
set.seed(1234)
x = rbinom(1000, 1, .6)
rle(x)
Run Length Encoding
  lengths: int [1:511] 1 5 2 1 1 1 2 1 1 1 ...
  values : int [1:511] 1 0 1 0 1 0 1 0 1 0 ...
max(rle(x)$len[rle(x)$val==0])
[1] 8

This same sequence of 1000 tosses happened to have a run of 10 Heads.
max(rle(x)$len[rle(x)$val==1])
[1] 10

A: Rather than counting tails, I would prefer to calculate a minimum sample size to conclude whether the coin may be biased. Here is one way this can be done, using Chebyshev's inequality. The following formula uses this inequality to get a sample size needed to estimate a mean that is within $\epsilon$ of the true one with probability $1-\delta$, given that we have an upper bound on the variance $\sigma^2$: $$ceil(\frac{\sigma^2}{\delta\epsilon^2}).$$
In this case, the variance $\sigma^2$ is bounded above by 1/4, which is the maximum variance a Bernoulli random variable can have, namely the variance for a fair coin ($(1/2)(1-1/2)$).
For example, we choose $\epsilon=1/10$ and $\delta=1/20$ (so we have a 95% confidence interval that the probability would be between 4/10 and 6/10 if the coin were fair), then we should flip the coin at least—$$t=ceil(\frac{\frac{1}{4}}{\frac{1}{20}*\frac{1}{10}^2}) = 500$$ times, count the number of heads, and divide by the number of flips to get the estimated probability of heads, which will be within 1/10 of the true probability at a 95% or greater chance.
Another way is Hoeffding's inequality. Using the same notation as before, the sample size for random variables in [0, 1] (including Bernoulli random variables) is— $$ceil(\frac{\ln(2/\delta)}{2\epsilon^2}), $$ so that the same example becomes— $$ceil(\frac{\ln(2/\frac{1}{20})}{2\frac{1}{10}^2}) = 185.$$
See also the reference below.
REFERENCES:

*

*Jiang, L., Hickernell, F.J., "Guaranteed Monte Carlo Methods for Bernoulli Random Variables", arXiv:1411.1151 [math.NA], 2014.

