3
$\begingroup$

In testing a global null hypothesis, with independent tests, the p-values are distributed as $U(0,1)$. There are many goodness-of-fit tests to check that the distribution is uniform. For example, this answer discusses the use of chi-squared, Kolmogorov-Smirnov, and several others.

However, in many articles about combining test results, I've seen the recommendation to use Fisher's combined test. Does it have any specific advantage for global null testing, compared to the more widely used tests of uniformity that I listed above?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ It's a good question. When thinking about it, be sure to consider (a) tests based on discrete statistics (such as counts) and (b) tests based on composite hypotheses (such as $H_0:\mu\le 0$ versus $H_A:\mu\gt 0$). I believe both of those situations provide counterexamples to your supposition of uniformly distributed p-values when the null is true. $\endgroup$
    – whuber ♦
    May 3 '16 at 0:56
3
$\begingroup$

The problem with the combination of $p$-values is that the question is usually not well specified. The methods available fall into two groups which are more sensitive to either (a) at least one $p_i$ is not from the uniform distribution (b) they all are not. Fisher's method is quite sensitive in situation (a) whereas Stouffer's method is sensitive in situation (b).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.