I have 30 p-values all very small (varying between $10^{-140}$ and $10^{-110}$), and I want to combine them in some way to get a single statistic. I learned about Fisher's method, but if I apply it to those small p-values, what I get is 0, probably due to lost of precision. But I want to get a certain number, not 0, since my purpose is to do the same for multiple methods and compare the statistics for each of them (each of the methods give 30 p-values, again all very small). I get 0 for each of the the methods, although they are in fact different, and getting 0 for all of the methods makes it impossible to compare them. A way is to get the mean of the logarithms of the 30 p-values, but I am not sure how much it makes sense. How can I solve this problem?
$\begingroup$
$\endgroup$
5
1 Answer
$\begingroup$
$\endgroup$
If underflow is the issue, you could simply compare the sum of their logs (the mean of the logs is entirely equivalent)
Note that Fishers method actually compares $-2\sum_{i=1}^k \log(p_i)$ to a $\chi^2_{2k}$ ... so Fisher would also be working on the log scale.
However, it's not clear to me that there's necessarily any particularly meaningful comparison between two sets of extremely small p-values. For starters, the values in the extreme tail will tend to be quite sensitive to even small deviations from assumptions.
Some people would argue that you shouldn't compare p-values at all.
pchisq(-2*sum(log(pvals)), 2*length(pvals), lower.tail=FALSE)in R. I think what I should do is to addlog.p=TRUEto the parameters ofpchisqfunction. $\endgroup$wilcox.testin R) and it gave those p-values. $\endgroup$