# How many flips are needed to confirm a coin's intentional bias?

I've made some 'special' coins and I know for a fact that they have a very small but known/designed bias $$\epsilon$$. I define bias as the probability $$(\frac{1}{2} + \epsilon)$$ towards heads and $$(\frac{1}{2} - \epsilon)$$ for tails. If someone were to test them, how many flips would they have to perform in order to measure $$\epsilon$$ with 95% confidence.

I know the exact bias (somehow), and expect the other party to measure it themselves and tell it back to me. The testing party has unlimited resources. $$\epsilon$$ is of the order of $$2^{-64}$$.

• 1. What exactly do you mean by the "bias" being $\epsilon$ exactly? A die has six sides, what population parameter(s) is this bias in relation to? 2. Given that the house edge is much larger that $\epsilon$, what use would a die with an extremely small bias be? Commented Aug 5, 2020 at 1:07
• @Glen_b I've defined bias in the edit. You've guessed that it's not a casino question. It's just easier phrased that way. I expect the number of rolls to be very very large, but would appreciate it quantifying :-) Commented Aug 5, 2020 at 1:28
• 1. If it's some other situation that doesn't involve six sides, I'd suggest using a model that comes closer to the actual circumstance. 2. Your definition is insufficient haven't yet explained how the deficit of $-\epsilon$ associated with the other 5 sides is disposed (this is not idle nitpicking; what alternative we're trying to detect affects probability of detection with some tests). If you only care about that $+\epsilon$ face (vs not-that-face) then a (possibly) biased die would be a more suitable model to discuss. Commented Aug 5, 2020 at 1:47
• @Bruce How did you come up with that? Isn't it the case that $1/4\times (2^{-64})^{-2} = 2^{126}$ tosses are needed to achieve a standard error of $2^{-64}?$
– whuber
Commented Aug 5, 2020 at 13:45
• For a 95% CI of the form $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}},$ 'Z' of the link is 1.96. So you'll be close using 2 instead. Then the formula is just $1/\Delta^2.$ In public opinion polling a result like $53\% \pm 2\%$ can result from polling $n = 2500$ subjects. $1/.02^2 = 2500.$ Commented Aug 5, 2020 at 16:26

It now sounds like you're after a sample size based on the width of a confidence interval of coverage $$1-\alpha$$. For a two sided interval of half-width $$h$$ (that is, the interval will be of the form ($$\hat\epsilon-h,\hat\epsilon+h)$$), the required sample size is:

$$n \geq Z_{α/2}^2 ­\pi(1-\pi) /h^2$$

where $$\pi$$ is the population proportion (for small $$\epsilon$$ relative to $$\pi$$ it won't matter whether you use $$\pi$$ or $$\pi+\epsilon$$).

If $$\alpha$$ is 0.05 (coverage 95%) the Z value is 1.96; this is often approximated by 2.

So if $$\pi$$ is $$\frac12$$ this is about $$1/h^2$$. If $$\pi$$ is even roughly in the ballpark of $$\frac12$$ this doesn't change much, but if it's getting close to 0 or 1, you will get smaller $$n$$.

If you want a one-sided interval (you care only about a lower bound on $$\epsilon$$, say) then the sample size reduces a little.

Hopefully this addresses your needs; strictly I should have continued asking questions to make sure I was answering the right question but this may be useful either way.

If you instead need a test with a given power to pick up a difference of $$\epsilon$$ that's a different calculation (though related).