# Fisher's method of combining p-values when one of the p-values is zero

I have a bunch of independent $p$-values and now I want to combine them using the Fisher's method. However, in R, when a $p$-value is zero, the log(p-value) becomes $-\infty$, so that the test statistic $X^2$ is $\infty$. This will give the Fisher's method $p$-value of 0.

If I check the $p$-values' histogram, they look pretty uniform, so I expect that the Fisher's method should give me a not-so-small $p$-value (should not reject the null). Is there a way to get around this issue? How about the Stouffer's method? Will that method automatically take care of the issue when $p = 0$ (or close to 0)?

Update: I tried the Stouffer's method, but still the issue of $\infty$ is not solved...

• Can you provide information on how you are calculating p-values? – Sameer May 9 '13 at 3:14
• Isn't the p-value just less than some number, which is based on the number of simulations you did? E.g. if you did 1000 sims, the p-value given is 0, but it's just <0.001. So you treat it as 0.001. (Or whatever). – Jeremy Miles May 9 '13 at 3:34
• @JeremyMiles When calculating bootstrap or Monte Carlo p-values if no test statistics as extreme as the observed statistic is found amongst the $B$ replicates, one generally sets the p-value to $\frac{1}{B}$ because the test statistic has been observed once (your observed test statistic!). However, this still doesn't account for the fact that you actually have a censored distribution i.e. you cannot observe p-values less than $\frac{1}{B}$. A quick Google search did not lead me anywhere; I wonder if someone else has thought about this problem. – Sameer May 9 '13 at 3:58
• @alittleboy re: "when r=N". According to your equation, when $r$ and $N$ are equal, $p=1$. Doing a 2-sided test doesn't mean calculating $p$, then calculating $1-p$, here, does it? – ndoogan May 9 '13 at 4:15
• @alittleboy, what are "r" & "N" here? You should not have any true 0's, even if you are doing a 2-sided test. – gung - Reinstate Monica Aug 12 '15 at 23:21

Irrespective of the discussion in the comments about how these $p$-values of $0$ arose there are methods for combining $p$-values which can be calculated if $p=0$.
The method of Edgington based on the sum of $p$, the closely related mean $p$ method, the method using logit of $p$, Tippett's method based on the minimum $p$ and variants of Wilkinson's method of which Tippett is a special case can all be calculated. Whether that is a sensible thing to do depends on the scientific question of course.