# How many times must I roll a die to confidently assess its fairness?

(Apologies in advance for use of lay language rather than statistical language.)

If I want to measure the odds of rolling each side of a specific physical six-sided die to within about +/- 2% with a reasonable confidence of certainty, how many sample die rolls would be needed?

i.e. How many times would I need to roll a die, counting each result, to be 98% sure that the chances it rolls each side are within 14.6% - 18.7%? (Or some similar criteria where one would be about 98% sure the die is fair to within 2%.)

(This is a real-world concern for simulation games using dice and wanting to be sure certain dice designs are acceptably close to 1/6 chance of rolling each number. There are claims that many common dice designs have been measured rolling 29% 1's by rolling several such dice 1000 times each.)

• This is a lot trickier than finding the confidence interval for a binomial, since you'd want to keep all probabilities in check. Have a look at Hsiuying Wang's paper on simultaneous confidence intervals for multinomial distributions (Journal of Multivariate Analysis 2008, 99, 5, 896-911). You can find some code in this blog post, which also gives a quick summary on some of the work that's been done on this. – idnavid Oct 9 '18 at 0:24
• Note that if you are just interested in checking whether 1's are rolled a fair amount of the time, this simplifies the question a lot. – Dennis Jaheruddin Oct 9 '18 at 11:10
• It is important to note that "confidence interval" does not give you a "percentile likelihood of being correct". I suspect that you are using the very reasonable common use of the term "98% sure", but you must know anytime someone mentions "confidence interval" that is not at all the same as a 98% likelyhood: link.springer.com/article/10.3758%2Fs13423-013-0572-3 – BrianH Oct 9 '18 at 13:55
• @BrianH Thank you! I did not just mean the colloquial expression, but am looking to quantify the certainty implied by the test. It seems to me that in the same way that it makes sense to say I expect to roll some die result a calculable percentage of the time, that there would be a similar (but more complex) calculation for how likely I am to roll results within a certain margin of error in I roll n times, which is what I think I understand Xiamoi's answer (and follow-up comment) is saying. Yes? – Dronz Oct 9 '18 at 16:09
• @Dronz To be fair, this is one of those things that you really think would be more straight-forward than it turns out to actually be. Devilishly tricky, in fact. Here's some key related questions elsewhere to help give you an idea of how there is no incredibly straight-forward answer: Frequentist math.stackexchange.com/questions/1578932/… Bayesian math.stackexchange.com/questions/1584833/… and fun: rpg.stackexchange.com/questions/70802/… – BrianH Oct 10 '18 at 19:46

TL;DR: if $$p$$ = 1/6 and you want to know how large $$n$$ needs to be 98% sure the dice is fair (to within 2%), $$n$$ needs to be at least $$n$$ ≥ 766.

Let $$n$$ be the number of rolls and $$X$$ the number of rolls that land on some specified side. Then $$X$$ follows a Binomial(n,p) distribution where $$p$$ is the probability of getting that specified side.

By the central limit theorem, we know that

$$\sqrt{n} (X/n - p) \to N(0,p(1-p))$$

Since $$X/n$$ is the sample mean of $$n$$ Bernoulli$$(p)$$ random variables. Hence for large $$n$$, confidence intervals for $$p$$ can be constructed as

$$\frac{X}{n} \pm Z \sqrt{\frac{p(1-p)}{n}}$$

Since $$p$$ is unknown, we can replace it with the sample average $$\hat{p} = X/n$$, and by various convergence theorems, we know the resulting confidence interval will be asymptotically valid. So we get confidence intervals of the form

$$\hat{p} \pm Z \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

with $$\hat{p} = X/n$$. I'm going to assume you know what $$Z$$-scores are. For example, if you want a 95% confidence interval, you take $$Z=1.96$$. So for a given confidence level $$\alpha$$ we have

$$\hat{p} \pm Z_\alpha \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

Now let's say you want this confidence interval to be of length less than $$C_\alpha$$, and want to know how big a sample we need to make this case. Well this is equivelant to asking what $$n_\alpha$$ satisfies

$$Z_\alpha \sqrt{\frac{\hat{p}(1-\hat{p})}{n_\alpha}} \leq \frac{C_\alpha}{2}$$

Which is then solved to obtain

$$n_\alpha \geq \left(\frac{2 Z_\alpha}{C_\alpha}\right)^2 \hat{p}(1-\hat{p})$$

So plug in your values for $$Z_\alpha$$, $$C_\alpha$$, and estimated $$\hat{p}$$ to obtain an estimate for $$n_\alpha$$. Note that since $$p$$ is unknown this is only an estimate, but asymptotically (as $$n$$ gets larger) it should be accurate.

• Thanks. As I have not done college-type math in decades, could I trouble you to plug in the numbers and actually give me a ballpark number of times I'd need to roll a die, as an integer? – Dronz Oct 9 '18 at 0:40
• if $p = 1/6$ and you want to know how large $n$ needs to be 98% sure the dice is fair to within 2%, $n$ needs to be at least $n \geq 766$. Ignore my last comment, used incorrect $C_\alpha$. – Xiaomi Oct 9 '18 at 1:40
• It might be more interesting to look at the multinomial distribution, since now we test for each side separately. This does not take into account all the information we have on the problem. For an intiuitive explanation look at stat.berkeley.edu/~stark/SticiGui/Text/chiSquare.htm – Jan Oct 9 '18 at 15:21
• I agree with @Jan: This answer does not address the question. Moreover, it cannot easily be adapted to construct an answer by applying it separately to all six faces, because the six tests are interdependent. – whuber Oct 9 '18 at 18:56
• This is a nice answer, but I fully agree with @Jan, whuber. This question deserves an answer based on chi-square statistic and multinomial distribution. – Łukasz Grad Oct 9 '18 at 19:49