# Significance of overlap between multiple lists

I am trying to evaluate the significance of overlap between several gene lists. Here I have applied different methods to select genes relevant to a disease and I have several 4 way venn diagrams illustrating the results.

My main goal is to determine whether the intersection of these 4 methods is significant so I can compare between each venn diagram.

To test the significance of overlap between two lists I would use a hypergeometric test however I cannot find any solutions to multiple overlapping problems.

Does someone know how I would achieve this?

• Could you explain more about why hypergeometric is the right distribution? It's not clear to me why what you describe is analogous to sampling without replacement. Apr 28, 2014 at 16:33

I'm dealing with similar problems, and haven't found a straightforward function. So I wrote a function myself. Although it's not very concise, it does the work. Hope it also helps you.

hyper_matrix <- function(gene.list, background){
# generate every combinations of two gene lists
combination <- expand.grid(names(gene.list),names(gene.list))
combination$values <- rep(NA, times=nrow(combination)) # convert long table into wide combination <- reshape(combination, idvar="Var1", timevar="Var2", direction="wide") rownames(combination) <- combination$Var1
combination <- combination[,-1]
colnames(combination) <- gsub("values.", "", colnames(combination))

# calculate the length of overlap of each pair
for(i in colnames(combination)){
for(j in rownames(combination)){
combination[j,i]<-length(intersect(gene.list[[j]],gene.list[[i]]))
}
}

# calculate the significance of the overlap of each pair
for(m in 1:length(gene.list)){
for(n in 1:length(gene.list)){
if(n>m){
combination[n,m] <- phyper(combination[m,n]-1, length(gene.list[[m]]), background-length(gene.list[[m]]), length(gene.list[[n]]), lower.tail=F)
# note that the phyper function (lower.tail=F) give the probability of P[X>x], so the the overlap length should subtract 1 to get a P[X>=x].
}
}
}
# round to 2 digit.
return(round(combination,2))
}


With this, if you have, let's say, 4 gene lists.

gene.list <- list(listA=paste0("gene",c(1,2,3,4,5,6,7,8,9)),
listB=paste0("gene",c(1,3,4,6,7,9)),
listC=paste0("gene",c(5,6,7,8,9,11)),
listD=paste0("gene",c(11,12,13,14,15)))


and the background number is 14 genes (number of all balls in the urn) in the world, the result would be:

hyper_matrix(gene.list, 14)

listA listB listC listD
listA  9.00  6.00  5.00     0
listB  0.03  6.00  3.00     0
listC  0.24  0.53  6.00     1
listD  1.00  1.00  0.97     5


where the upper triangle on the right is the lengths of overlap of each pair, and the bottom triangle on the left is the significance of the overlap by hypergeometric test. Here in this toy example, the overlap between listA and listB is significant in a world a 14 genes if you choose 0.05 as your p-value cutoff. Any other pair is not significantly overlapping.