Skip to main content
added 237 characters in body; added 13 characters in body; added 30 characters in body; added 113 characters in body
Source Link
Paul
  • 1.4k
  • 1
  • 16
  • 16

If you have 1,000,000 independent "coin flips" that can produce 1 with probabilty (prob) and 0 with probability (1-prob), then the number of 1's observed will follow a Binomial distribution.

Tests of statistical significance are rejection tests, i.e. reject the hypothesis that the two parameters are equal if the probability that param2 is observed in test2 when the true value is param1 is less than a certain number, like 5%, 1%, or 0.1%. These tests are typically constructed from the cumulative distribution function.

The cumulative distribution function for a binomial is ugly, but can be found in R and probably some other statistics packages as well.

But the good news is that with 1,000,000 cases you don't need to do that.... you would if you had a relatively small number of cases.

Because you have 1,000,000 independent flips, the CDF of a normal distribution is a good approximation (the Law of Large Numbers plays a role here). The mean and variance you need to use are the obvious ones, and are in the Binomial Wikipedia article... You are then comparing two normally distributed variables and can use all the standard tests you would use with normally distributed variables.

For instance, if the true probability were 40*10^-6 then in 1,000,000 tests you would expect to see 40 +/- 6 positive cases. If the acceptance interval for a test is, for instance, 5 standard deviations wide on each side, then this would be compatible with both observations. If it were just 3 std dev wide on each side, one case would fit and the other would be statistically different.

If you have 1,000,000 independent "coin flips" that can produce 1 with probabilty (prob) and 0 with probability (1-prob), then the number of 1's observed will follow a Binomial distribution.

Tests of statistical significance are rejection tests, i.e. reject the hypothesis that the two parameters are equal if the probability that param2 is observed in test2 when the true value is param1 is less than a certain number, like 5%, 1%, or 0.1%. These tests are typically constructed from the cumulative distribution function.

The cumulative distribution function for a binomial is ugly, but can be found in R and probably some other statistics packages as well.

But the good news is that with 1,000,000 cases you don't need to do that.... you would if you had a relatively small number of cases.

Because you have 1,000,000 independent flips, the CDF of a normal distribution is a good approximation (the Law of Large Numbers plays a role here). The mean and variance you need to use are the obvious ones, and are in the Binomial Wikipedia article... You are then comparing two normally distributed variables and can use all the standard tests you would use with normally distributed variables.

If you have 1,000,000 independent "coin flips" that can produce 1 with probabilty (prob) and 0 with probability (1-prob), then the number of 1's observed will follow a Binomial distribution.

Tests of statistical significance are rejection tests, i.e. reject the hypothesis that the two parameters are equal if the probability that param2 is observed in test2 when the true value is param1 is less than a certain number, like 5%, 1%, or 0.1%. These tests are typically constructed from the cumulative distribution function.

The cumulative distribution function for a binomial is ugly, but can be found in R and probably some other statistics packages as well.

But the good news is that with 1,000,000 cases you don't need to do that.... you would if you had a relatively small number of cases.

Because you have 1,000,000 independent flips, the CDF of a normal distribution is a good approximation (the Law of Large Numbers plays a role here). The mean and variance you need to use are the obvious ones, and are in the Binomial Wikipedia article... You are then comparing two normally distributed variables and can use all the standard tests you would use with normally distributed variables.

For instance, if the true probability were 40*10^-6 then in 1,000,000 tests you would expect to see 40 +/- 6 positive cases. If the acceptance interval for a test is, for instance, 5 standard deviations wide on each side, then this would be compatible with both observations. If it were just 3 std dev wide on each side, one case would fit and the other would be statistically different.

Source Link
Paul
  • 1.4k
  • 1
  • 16
  • 16

If you have 1,000,000 independent "coin flips" that can produce 1 with probabilty (prob) and 0 with probability (1-prob), then the number of 1's observed will follow a Binomial distribution.

Tests of statistical significance are rejection tests, i.e. reject the hypothesis that the two parameters are equal if the probability that param2 is observed in test2 when the true value is param1 is less than a certain number, like 5%, 1%, or 0.1%. These tests are typically constructed from the cumulative distribution function.

The cumulative distribution function for a binomial is ugly, but can be found in R and probably some other statistics packages as well.

But the good news is that with 1,000,000 cases you don't need to do that.... you would if you had a relatively small number of cases.

Because you have 1,000,000 independent flips, the CDF of a normal distribution is a good approximation (the Law of Large Numbers plays a role here). The mean and variance you need to use are the obvious ones, and are in the Binomial Wikipedia article... You are then comparing two normally distributed variables and can use all the standard tests you would use with normally distributed variables.