# How many tests are needed in order to be x% confident that a predicted probability set is correct?

Let $O$ be a set of outcomes $O_1, O_2, ... O_n$ with [positive] probabilities $P_1, P_2, ... P_n$ in $P$, with $\sum P_i = 1$. How many outcomes, $f(K)$, would you need to be $k$% sure that the probability distribution described by set $P$ is correct?

If it's unclear what I mean, here's an example question:

Suppose I have a gumball machine, and on the owner's manual it says that the machine dispenses red gumballs 15% of the time, yellow gumballs 15% of the time, green gumballs 30% of the time, and blue gumballs 40% of the time.

How many gumballs would I need as (a function of x) such that I could be x% sure of the above distribution probabilities?

Sorry if this is a simple question. The only statistics I have done are simple z-scores, t-tests and the like.

Edit: By "x% confident," I mean that one would expect x% of confidence intervals for the probabilities of the events to contain the true probabilities.

• To be certain (as per your title), you need the entire population. It's unclear what "$k\%$ sure" and "x% sure" is intended to mean in practice. – Glen_b Aug 19 '17 at 4:58
• Certain means 100% confidence, yes? Surely you wouldn't need the whole population at, say, a 50% confidence level. – Tiwa Aina Aug 19 '17 at 5:00
• Read your title, which talks about being "certain". I don't know what you intend by "100% confidence" (nor 50% confidence). Note that in statistics, the word "confidence" carries a particular meaning that I doubt you intend here --- either that or you're missing information which would give it meaning. – Glen_b Aug 19 '17 at 5:00
• Oops, I see what you mean. I just changed the title to make it more appropriate for my question. – Tiwa Aina Aug 19 '17 at 5:02
• Your title and body text are now more consistent but it's still unclear what "x% confident", "$k\%$ sure" and "x% sure" is intended to mean in practice. If you're intending to use these terms in their statistical sense your request makes no sense and if you're not intending these terms in their statistical sense I don't get what you actually want. – Glen_b Aug 19 '17 at 5:03