2
$\begingroup$

Example: I want to compare the number of successes in a sample of ~10,000,000 independent bernoulli trials against a known population probability of success of ~.01. Thus, expected success count for 10,000,000 trials should be 100,000.

What's the quickest hypothesis test to implement for a large n, small p? I would prefer to avoid the exact binomial test if possible as I'm trying to code an automated hypothesis test.

|cite|improve this question
$\endgroup$
1
$\begingroup$

From the Wikipedia article on the Binomial test,

For large samples such as the example below, the binomial distribution is well approximated by convenient continuous distributions, and these are used as the basis for alternative tests that are much quicker to compute, Pearson's chi-squared test and the G-test. However, for small samples these approximations break down, and there is no alternative to the binomial test.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ makes sense; now it's a matter of determining practical significance of test. $\endgroup$ – benrolls Mar 16 '12 at 15:31
  • $\begingroup$ wait, with a small p, can you really approximate with a normal distribution? In reference to this wiki on 2 cell chi-square test:en.wikipedia.org/wiki/Pearson%27s_chi-squared_test#Two_cells $\endgroup$ – benrolls Mar 16 '12 at 15:51
  • $\begingroup$ I think it depends on just how large N is relative to p, but that it can be done. You're correct that it requires care to be sure, though. $\endgroup$ – ely Mar 16 '12 at 16:12
  • 1
    $\begingroup$ @benrolls and EMS>> In this case the standard deviation under the null is 315, meaning if the null is true you'd expect about 95% of runs to give resulting in the range $1.0 \cdot 10^5 \pm 730$. The normal approximation will be fine. (In fact, if you're happy with a 5% significance level, that's your hypothesis test right there.) $\endgroup$ – Cyan Mar 16 '12 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.