How to test if a subset of point is localized in the tail of distribution I've a bunch of genes (~8000). Each of these genes has an associated p-value representing its "importance" in a specific biological pathway. Within these genes I'm interested in a subset of genes (~200). How can I test wheter these 200 genes are in the tail of the p-value distribution. In other terms, how can I test wheter these 200 genes are "more" significant than the rest of the genes. For now I did an hypergeometric test using a thresold on the p-value. Let's say :
A : Total genes (~8000)
B : Subset of gene of interest (~200)
C : Genes in A with p-values <= 0.05
D : Genes in B with p-values <= 0.05 

and then
phyper(D,C,A-C,B)

but I have to define a thresold (0.05) ....
Edit : Can I do a Mann-Whitney U test on the p-values ? (comparing A vs B ; by removing the subset B from A of course)
Thanks
 A: Different possibilities come to mind.


*

*You could run a two-sample Kolmogorov-Smirnov test to check whether both p value distributions come from the same population. This would also come out significant of the smaller set is systematically larger than the rest, though.

*You could do a simple one-sided two-sample unpaired t test (without assuming equal variance) to check whether one sample's mean is lower than the other.

*If you want to test against specific violations of uniformity, you could use Neyman's smooth test - the different components "look at" different sections of the unit interval. Ledwina (1994, JASA) offers a way to select the number of components based on BIC - it's implemented in the ddst package for R.

*Plotting the empirical cumulative distribution function of both samples against each other would certainly be informative (this is how the Kolmogorov-Smirnov test works) - it wouldn't be a test, though. I am a bit unsure about how valid it is to test the distributions of p values against each other the way you propose, so I personally would trust such an ECDF plot more than some test, anyway.
