I'm comparing success rates for two repeated experiments with $n_1$ and $n_2$ successes out of $N$ trials. $N$ is in the order of $10^7$. I don't know beforehand which experiment has a lower success rate, so I'd probably use a two-sided test. I see two options:
- The test of equal proportions (in R:
prop.test(c(n1, n2), c(N, N)))
- Fisher's exact test (in R:
fisher.test(matrix(c(n1, n2, N-n1, N-n2), ncol = 2)))
Now, for $N$ that large, Fisher's exact test is slow. (The implementation seems to evaluate the density of the hypergeometric distribution on a support of the order of $N$.) However, the test of equal proportions seems to have less power.
Does the test of equal proportions always return a p-value not less than that of Fisher's exact test for the same data? Is there a more powerful alternative to the test of equal proportions that isn't that expensive computation-wise?
EDIT: A computational test on 1000 matrices with "almost equal" entries suggests that the p-value computed by Fisher's test is almost always, but not always, larger. I'm still looking for stronger argumentation, and a faster/more powerful test.