I am doing a two tailed exact binomial goodness of fit test, see, e.g., here. I am looking for something similar as for Chi-square tests, which, as a rule of thumb, require a frequency of 5 for each cell in the table. So, what is the minimum sample size for an exact binomial test of goodness of fit (two-tailed)?


1 Answer 1

  1. You have the "rule of thumb" wrong. It doesn't say you need a frequency of 5 in each cell. It relates to expected frequency under the null hypothesis. You could have 10 in one cell and 0 in the other, but if $p_0=\frac12$ then the expected frequencies are both 5.

  2. The point of the rule of thumb is that then the discrete distribution of the chi-squared statistic should be reasonably close to the continuous chi-squared distribution you're looking up when you do a chi-squared test.

    Here's the null distribution (i.e. cdf) of the (discrete) chi-square statistic in the above situation (n=10,p=$\frac{_1}{^2}$) (blue) and the continuous chi-square distribution that approximates it:

    enter image description here

    [If combined with Yates' continuity correction that would give a better approximation.]

  3. With the exact binomial test you're looking up what will be* the exact discrete distribution of the count in one cell, so there's no minimum sample size at which it applies, since you're not dealing with an approximation.

* under the assumptions of independence and constant probability per trial

  • $\begingroup$ I meant to say 'expected frequency'. When I am doing a two-tailed exact binomial test of goodness of fit with p_0 =1/2, I need observed frequencies of 6 and 0, respectively, to obtain a p value below 0.05. Given this threshold, does that mean I need at least 6 observations to make any claims on statistical significance? $\endgroup$
    – You_got_it
    Commented Dec 3, 2015 at 1:58
  • $\begingroup$ That depends on your significance level; if you don't restrict yourself to a type I error rate no larger than 0.05, you can make the claim at smaller sample sizes. For example with n=5 you can achieve significance at the 0.0625 level. There's nothing sacrosanct about 0.05. $\endgroup$
    – Glen_b
    Commented Dec 3, 2015 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.