I wanna make t-test for effect of gender, another t-test for effect of age, one-way ANOVA for effect of education and MIXED ANOVA (3x8) for another effect? Should I use Bonferroni corection? All is on same dependent variable.


The key point about the multiple testing problem is that every time you have a p-value you have to pay -- like rolling the dice in a casino. A table of p-values needs to be adjusted by the number of tests. Bonferroni is very conservative (i.e., will really lower the adjusted p-values) so you may want to invoke the false discovery rate (FDR) instead -- look up the 1995 paper by Benjamini & Hochberg. But yes, for 2-way and multiway ANOVA, most labs apply a Bonferroni correction.

  • $\begingroup$ There is also bootstrap and permutation p-value adjustment methods. This is covered in the book by Westfall and Young. "Resampling-Based Multiple Testing Examples and Methods for p-value Adjustment" Peter H. Westfall and S. Stanley Young, Wiley 1993. $\endgroup$ Dec 18 '16 at 1:31
  • $\begingroup$ Here is a URL for it. www.wiley.com/WileyCDA/WileyTitle/productCd-0471557617.html $\endgroup$ Dec 18 '16 at 1:34
  • $\begingroup$ The Westfall & Young procedure is also a good option. $\endgroup$
    – user32398
    Dec 18 '16 at 1:46
  • $\begingroup$ I noticed that Peter Westfall is also a member on CV. The advantage of the bootstrap approach is that it is not nearly as conservative as Bonferroni. I am pretty sure it deals with family-wise error rate and not False Discovery Rate. $\endgroup$ Dec 18 '16 at 2:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.