Skip to main content
1 of 2
Paul
  • 1.4k
  • 1
  • 16
  • 16

If you have 1,000,000 independent "coin flips" that can produce 1 with probabilty (prob) and 0 with probability (1-prob), then the number of 1's observed will follow a Binomial distribution.

Tests of statistical significance are rejection tests, i.e. reject the hypothesis that the two parameters are equal if the probability that param2 is observed in test2 when the true value is param1 is less than a certain number, like 5%, 1%, or 0.1%. These tests are typically constructed from the cumulative distribution function.

The cumulative distribution function for a binomial is ugly, but can be found in R and probably some other statistics packages as well.

But the good news is that with 1,000,000 cases you don't need to do that.... you would if you had a relatively small number of cases.

Because you have 1,000,000 independent flips, the CDF of a normal distribution is a good approximation (the Law of Large Numbers plays a role here). The mean and variance you need to use are the obvious ones, and are in the Binomial Wikipedia article... You are then comparing two normally distributed variables and can use all the standard tests you would use with normally distributed variables.

Paul
  • 1.4k
  • 1
  • 16
  • 16