I am talking about a hypergeometric test but the logic probably applies as well to the binomial test (and other similar tests).

Basic Scenario

I have a population of $n+m$ balls in an urn. There are $m$ white balls and $n$ black balls. I draw a sample of $k$ balls without replacement. I get $q$ white balls and $k-q$ black balls.

  • Null hypothesis: White and black balls have the same probability of being sampled.

  • Alternative hypothesis: White and black balls DO NOT have the same probability of being sampled.

One-tailed hypergeometric test

As there is no replacement, the probability of obtaining $q$ white balls is given by a hypergeometric distribution. I can calculate the probability of getting $k$ or less than $k$ balls in my sample by adding all the probability over the range k to 0. In other words, I can compute $\sum_{i=0}^q P(i)$, where $P(i)$ is the probability of of drawing $i$ balls given by the hypergeometric function. As I am using R, here is the R code for this calculation phyper(q=q, m=m, n=n, k=k).

Similarly, I could compute the probability of obtaining $q$ or more balls: $\sum_{i=q}^k P(i)$. In R it gives phyper(q=q-1, m=m, n=n, k=k, lower.tail = FALSE)

My question is...

How can I get a p.value for a two-tailed hypergeometric test?

From wiki I read about two-tailed binomial test:

There are two methods to define the two tailed p-value. One method is to sum the probability that the total deviation in numbers of events in either direction from the expected value is either more than or less than the expected value. [..] The second method involves computing the probability that the deviation from the expected value is as unlikely or more unlikely than the observed value, i.e. from a comparison of the probability density functions

I think this is very related to what I need but I don't quite get it!


1 Answer 1


I ended up considering the mass of probability on the shortest tail and multiplied this probability by 2 to account for the fact that it is a two-tailed test.

I have no reference that this is the best method but it felt quite intuitive to me!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.