Significance of overlap between multiple lists

Question

I am trying to evaluate the significance of overlap between several gene lists. Here I have applied different methods to select genes relevant to a disease and I have several 4 way venn diagrams illustrating the results.

My main goal is to determine whether the intersection of these 4 methods is significant so I can compare between each venn diagram.

To test the significance of overlap between two lists I would use a hypergeometric test however I cannot find any solutions to multiple overlapping problems.

Does someone know how I would achieve this?

Answer 1

I'm dealing with similar problems, and haven't found a straightforward function. So I wrote a function myself. Although it's not very concise, it does the work. Hope it also helps you.

hyper_matrix <- function(gene.list, background){
  # generate every combinations of two gene lists
  combination <- expand.grid(names(gene.list),names(gene.list))
  combination$values <- rep(NA, times=nrow(combination))

  # convert long table into wide
  combination <- reshape(combination, idvar="Var1", timevar="Var2", direction="wide")
  rownames(combination) <- combination$Var1
  combination <- combination[,-1]
  colnames(combination) <- gsub("values.", "", colnames(combination))

  # calculate the length of overlap of each pair
  for(i in colnames(combination)){
    for(j in rownames(combination)){
      combination[j,i]<-length(intersect(gene.list[[j]],gene.list[[i]]))
    }
  }

  # calculate the significance of the overlap of each pair
  for(m in 1:length(gene.list)){
    for(n in 1:length(gene.list)){
      if(n>m){
        combination[n,m] <- phyper(combination[m,n]-1, length(gene.list[[m]]), background-length(gene.list[[m]]), length(gene.list[[n]]), lower.tail=F)
        # note that the phyper function (lower.tail=F) give the probability of P[X>x], so the the overlap length should subtract 1 to get a P[X>=x].
      }
    }
  }
  # round to 2 digit.
  return(round(combination,2))
}

With this, if you have, let's say, 4 gene lists.

gene.list <- list(listA=paste0("gene",c(1,2,3,4,5,6,7,8,9)),
                 listB=paste0("gene",c(1,3,4,6,7,9)),
                 listC=paste0("gene",c(5,6,7,8,9,11)),
                 listD=paste0("gene",c(11,12,13,14,15)))

and the background number is 14 genes (number of all balls in the urn) in the world, the result would be:

hyper_matrix(gene.list, 14)

      listA listB listC listD
listA  9.00  6.00  5.00     0
listB  0.03  6.00  3.00     0
listC  0.24  0.53  6.00     1
listD  1.00  1.00  0.97     5

where the upper triangle on the right is the lengths of overlap of each pair, and the bottom triangle on the left is the significance of the overlap by hypergeometric test. Here in this toy example, the overlap between listA and listB is significant in a world a 14 genes if you choose 0.05 as your p-value cutoff. Any other pair is not significantly overlapping.