Example: I want to compare the number of successes in a sample of ~10,000,000 independent bernoulli trials against a known population probability of success of ~.01. Thus, expected success count for 10,000,000 trials should be 100,000.

What's the quickest hypothesis test to implement for a large n, small p? I would prefer to avoid the exact binomial test if possible as I'm trying to code an automated hypothesis test.


1 Answer 1


From the Wikipedia article on the Binomial test,

For large samples such as the example below, the binomial distribution is well approximated by convenient continuous distributions, and these are used as the basis for alternative tests that are much quicker to compute, Pearson's chi-squared test and the G-test. However, for small samples these approximations break down, and there is no alternative to the binomial test.

  • $\begingroup$ makes sense; now it's a matter of determining practical significance of test. $\endgroup$
    – benrolls
    Commented Mar 16, 2012 at 15:31
  • $\begingroup$ wait, with a small p, can you really approximate with a normal distribution? In reference to this wiki on 2 cell chi-square test:en.wikipedia.org/wiki/Pearson%27s_chi-squared_test#Two_cells $\endgroup$
    – benrolls
    Commented Mar 16, 2012 at 15:51
  • $\begingroup$ I think it depends on just how large N is relative to p, but that it can be done. You're correct that it requires care to be sure, though. $\endgroup$
    – ely
    Commented Mar 16, 2012 at 16:12
  • 1
    $\begingroup$ @benrolls and EMS>> In this case the standard deviation under the null is 315, meaning if the null is true you'd expect about 95% of runs to give resulting in the range $1.0 \cdot 10^5 \pm 730$. The normal approximation will be fine. (In fact, if you're happy with a 5% significance level, that's your hypothesis test right there.) $\endgroup$
    – Cyan
    Commented Mar 16, 2012 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.