Wikipedia says that p-values are uniformly distributed over [0,1] if the null hypothesis is true and for continuous data.

What is the expected p-value distribution if the test statistic is discrete? E.g. occurrences in an enrichment analysis.

  • 2
    $\begingroup$ See the discussion at Non-uniform distribution of p-values when simulating binomial tests under the null hypothesis -- a step function that touches the $y=x$ line at the top of each step is the general case. I suggest there that it could be called quasi-uniform while whuber has argued in comments elsewhere on site that is should just be called 'discrete uniform'. However, that doesn't match a fairly conventional usage of the term, so I don't think that suggestion will catch on. $\endgroup$ – Glen_b Oct 19 '17 at 9:20
  • 2
    $\begingroup$ Note that it's not the data but the distribution of the test statistic that is at issue. Consider, for example, a two-sample Kolmogorov-Smirnov test; the data can be continuous but the discreteness of the distribution of the test statistic is the issue. I have edited the question accordingly. $\endgroup$ – Glen_b Oct 19 '17 at 9:27
  • $\begingroup$ Another (very common) situation in which the Wikipedia claim is false is a "composite null" hypothesis, such as testing $H_0:\mu \le 0$ against the alternative $H_A:\mu \gt 0$ for data from a Normal$(\mu,\sigma^2)$ distribution. The p-value will be uniformly distributed only when $\mu=0$; if $\mu\lt 0$ (which is still part of the null hypothesis), the p-value will be concentrated closer to $1$. $\endgroup$ – whuber Oct 19 '17 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.