2
$\begingroup$

I'm learning about combining $p$-values and I have a couple of (somewhat elementary, perhaps) questions. Suppose I have $n$ independent tests, each of these to test its own null hypothesis $H_0 (i)$, $i \in \{1, \ldots, n\}$. Also suppose that, for each of these independent tests, I must calculate a test statistic $T_i \sim \chi_1^2$ (under the null hypothesis). My questions are:

(1) What's the main difference between calculating $\sum_{i=1}^n T_i \sim \chi_n^2$ (and then obtaining its $p$-value) and using Fisher's method?

(2) Would the global null hypothesis be "We can't reject any individual null hypothesis" and the global alternative hypothesis "We can reject at least one individual null hypothesis"?

$\endgroup$
2
$\begingroup$

Fisher's method involves converting each $p$-value into a $\chi^2_2$ and then summing them. Since there is a simple formula for doing this he may, given the technology of his time, have preferred this to what you suggest in your first question. There is not a fundamental difference between them though.

The whole problem is quite poorly specified. The null is that all the $p$s are drawn from a uniform distribution on (0, 1). The alternative is either that (a) at least one of them is not (b) none of them is. For more details consult

@article{birnbaum54,
   author = {Birnbaum, A},
   title = {Combining independent tests of significance},
   journal = {Journal of the American Statistical Association},
   year = {1954},
   volume = {49},
   pages = {559--574},
   keywords = {meta-analysis, significance values}
}

and

@misc{cousins08,
   author = {Cousins, R D},
   title = {Annotated bibliography of some papers on combining
      significances or $p$--values},
   year = {2008},
   note = {arXiv:0705.2209},
   keywords = {meta-analysis, significance values}
}
| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ +1, interested readers can also see stats.stackexchange.com/questions/78596, there are some nice illustrations in that thread. I also mentioned Cousins paper there, I recall it's a decent overview. $\endgroup$ – amoeba Aug 5 '16 at 14:46
  • $\begingroup$ @amoeba I think that must have been where I found Cousin's paper, thanks for the link $\endgroup$ – mdewey Aug 5 '16 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.