2
$\begingroup$

I'm learning about combining $p$-values and I have a couple of (somewhat elementary, perhaps) questions. Suppose I have $n$ independent tests, each of these to test its own null hypothesis $H_0 (i)$, $i \in \{1, \ldots, n\}$. Also suppose that, for each of these independent tests, I must calculate a test statistic $T_i \sim \chi_1^2$ (under the null hypothesis). My questions are:

(1) What's the main difference between calculating $\sum_{i=1}^n T_i \sim \chi_n^2$ (and then obtaining its $p$-value) and using Fisher's method?

(2) Would the global null hypothesis be "We can't reject any individual null hypothesis" and the global alternative hypothesis "We can reject at least one individual null hypothesis"?

$\endgroup$

1 Answer 1

2
$\begingroup$

Fisher's method involves converting each $p$-value into a $\chi^2_2$ and then summing them. Since there is a simple formula for doing this he may, given the technology of his time, have preferred this to what you suggest in your first question. There is not a fundamental difference between them though.

The whole problem is quite poorly specified. The null is that all the $p$s are drawn from a uniform distribution on (0, 1). The alternative is either that (a) at least one of them is not (b) none of them is. For more details consult

@article{birnbaum54,
   author = {Birnbaum, A},
   title = {Combining independent tests of significance},
   journal = {Journal of the American Statistical Association},
   year = {1954},
   volume = {49},
   pages = {559--574},
   keywords = {meta-analysis, significance values}
}

and

@misc{cousins08,
   author = {Cousins, R D},
   title = {Annotated bibliography of some papers on combining
      significances or $p$--values},
   year = {2008},
   note = {arXiv:0705.2209},
   keywords = {meta-analysis, significance values}
}
$\endgroup$
2
  • 1
    $\begingroup$ +1, interested readers can also see stats.stackexchange.com/questions/78596, there are some nice illustrations in that thread. I also mentioned Cousins paper there, I recall it's a decent overview. $\endgroup$
    – amoeba
    Commented Aug 5, 2016 at 14:46
  • $\begingroup$ @amoeba I think that must have been where I found Cousin's paper, thanks for the link $\endgroup$
    – mdewey
    Commented Aug 5, 2016 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.