How do I calculate if the degree of overlap between two lists is significant?

Question

If I have two lists A and B, both of which are subsets of a much larger list C, how can I determine if the degree of overlap of A and B is greater than I would expect by chance?

Should I just randomly select elements from C of the same lengths as lists A and B and determine that random overlap, and do this many times to determine some kind or empirical p-value? Is there a better way to test this?

Answer 1

If I understand your question correctly, you need to use the Hypergeometric distribution. This distribution is usually associated with urn models, i.e there are $n$ balls in an urn, $y$ are painted red, and you draw $m$ balls from the urn. Then if $X$ is the number of balls in your sample of $m$ that are red, $X$ has a hyper-geometric distribution.

For your specific example, let $n_A$, $n_B$ and $n_C$ denote the lengths of your three lists and let $n_{AB}$ denote the overlap between $A$ and $B$. Then

$$n_{AB} \sim \text{HG}(n_A, n_C, n_B)$$

To calculate a p-value, you could use this R command:

#Some example values
n_A = 100;n_B = 200; n_C = 500; n_A_B = 50
1-phyper(n_A_B, n_B, n_C-n_B, n_A)
[1] 0.008626697

Word of caution. Remember multiple testing, i.e. if you have lots of A and B lists, then you will need to adjust your p-values with a correction. For the example the FDR or Bonferroni corrections.

Answer 2

csgillespie's answer seems correct except for one thing: it gives the probability of seeing strictly more than n_A_B in the overlap, P(x > n_A_B), but I think OP wants the pvalue P(x >= n_A_B). You could get the latter by

n_A = 100;n_B = 200; n_C = 500; n_A_B = 50
phyper(n_A_B - 1, n_A, n_C-n_A, n_B, lower.tail = FALSE)