Suppose, I have done:

  • $n_1$ independent trials with an unknown success rate $p_1$ and observed $k_1$ successes.
  • $n_2$ independent trials with an unknown success rate $p_2$ and observed $k_2$ successes.

If, now $p_1 = p_2 =: p$ but still unknown, the probability $p(k_2)$ to observe $k_2$ for a given $k_1$ (or vice versa) is proportional to $\int_0^1 B(n_1,p,k_1) B(n_2, p, k_2) \text{d}p = \frac{1}{n_1+n_2+1}\binom{n_1}{k_1}\binom{n_2}{k_2}\binom{n_1+n_2}{k_1+k_2}^{-1}$, so if I want to test for $p_1 \neq p_2$, I only need to look in which quantile of the corresponding distribution my observations are.

So far for reinventing the wheel. Now my problem is that I fail to find this in literature, and thus I wish to know: What is the technical term for this test or something similar?

  • 2
    $\begingroup$ Why not use the two-proportion z-test (en.wikipedia.org/wiki/Statistical_hypothesis_testing) (If I understand your problem correctly). $\endgroup$ – Verena Haunschmid Dec 19 '13 at 15:04
  • $\begingroup$ @ExpectoPatronum: At a quick glance the biggest problem is that this test requires at least 5 successes and failures for each observation, which may not be given in my application and also indicates that (unneccessary) approximations are made. $\endgroup$ – Wrzlprmft Dec 19 '13 at 16:20
  • $\begingroup$ ok, that is a problem but most tests have similar requirements. $\endgroup$ – Verena Haunschmid Dec 19 '13 at 16:26
  • $\begingroup$ @ExpectoPatronum: Anyway searching for an exact alternative to the two-proportion z-test, I found Fisher’s exact test, which looks very similar at first glance (but I have yet to look into it in detail). $\endgroup$ – Wrzlprmft Dec 19 '13 at 16:31
  • 1
    $\begingroup$ @ExpectoPatronum: The division does not matter, since the large term is only proportional to $p(k_2)$ and $(n_1+n_2+1)$ is exactly the normalisation constant. Anyway, I have now confirmed that this is Fisher’s Exact Test, which I found thanks to you. $\endgroup$ – Wrzlprmft Dec 19 '13 at 21:41

The test statistics $p(k_2)$ is that of Fisher’s Exact Test.

Since $$\sum_{k_2}^{n_2} \frac{1}{n_1+n_2+1}\binom{n_1}{k_1}\binom{n_2}{k_2}\binom{n_1+n_2}{k_1+k_2}^{-1} = \frac{1}{n_1+n_2+1},$$ normalisation can be obtained by multiplying with $n_1+n_2+1$ and thus: $$p(k_2) = \binom{n_1}{k_1}\binom{n_2}{k_2}\binom{n_1+n_2}{k_1+k_2}^{-1}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.