I am using a ranksum test to compare the median of two samples ($n=120000$) and have found that they are significantly different with: p = 1.12E-207. Should I be suspicious of such a small $p$-value or should I attribute it to the high statistical power associated with having a very large sample? Is there any such thing as a suspiciously low $p$-value?


3 Answers 3


P-values on standard computers (using IEEE double precision floats) can get as low as approximately $10^{-308}$. These can be legitimately correct calculations when effect sizes are large and/or standard errors are low. Your value, if computed with a T or normal distribution, corresponds to an effect size of about 31 standard errors. Remembering that standard errors usually scale with the reciprocal square root of $n$, that reflects a difference of less than 0.09 standard deviations (assuming all samples are independent). In most applications, there would be nothing suspicious or unusual about such a difference.

Interpreting such p-values is another matter. Viewing a number as small as $10^{-207}$ or even $10^{-10}$ as a probability is exceeding the bounds of reason, given all the ways in which reality is likely to deviate from the probability model that underpins this p-value calculation. A good choice is to report the p-value as being less than the smallest threshold you feel the model can reasonably support: often between $0.01$ and $0.0001$.

  • 14
    $\begingroup$ When I reported ''$p<10^{-26}$'' in a conference paper, a reviewer told me that I should change it to ''$p<0.001$'' in order to follow APA guidelines. $\endgroup$ Jun 10, 2011 at 23:15
  • 4
    $\begingroup$ @whuber - Beautifully stated. $\endgroup$
    – rolando2
    Jun 11, 2011 at 4:24
  • 2
    $\begingroup$ (+1) At some point it's more likely that the government is nefariously flipping bits in your RAM remotely with super spy technology... $\endgroup$
    – JMS
    Jun 11, 2011 at 4:56
  • 5
    $\begingroup$ (+1) You can actually get down to just below $5 \times 10^{-324}$ in IEEE double precision floating point. But, your numerical routines for calculating $p$-values are almost guaranteed to fall apart before then. Unless you know for a fact that your modeling assumptions are perfectly correct (and when are they?), a $p$-value eventually just becomes a measure of the sample size once the sample gets large enough. $\endgroup$
    – cardinal
    Jun 11, 2011 at 15:18
  • 2
    $\begingroup$ I agree with @naught: especially for those trying to reproduce or check a paper, having a reasonably precise value for comparison is helpful. Changing "$\lt 10^{-26}$" to "$\lt 0.001$" erases 23 significant digits! Evidently, the APA guidelines focus on reporting results rather than on checking them -- to the detriment of all science. $\endgroup$
    – whuber
    Aug 3, 2021 at 15:06

There is nothing suspicious -- extremely low p-values like yours are pretty common when sample sizes are large (as yours is for comparing medians). As whuber mentioned, normally such p-values are reported as being less than some threshold (e.g. <0.001).

One thing to be careful about is that p-values only tells you whether the difference in median is is statistically significant. Whether the difference is significant enough in magnitude is something you will have to decide: e.g. for large sample sets, extremely small differences in means/medians can be statistically significant, but it might not mean very much.


A p-value can achieve a value of 0.

Suppose I am testing the composite hypothesis about the value of a range of a uniform 0, $\theta$ random variable. If I set $\mathcal{H}_0: \theta = 1$ and sample a value of $X=1.1$, you see it's impossible to observe such a value or higher under the null hypothesis. The p-value is 0.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.