I wanna make t-test for effect of gender, another t-test for effect of age, one-way ANOVA for effect of education and MIXED ANOVA (3x8) for another effect? Should I use Bonferroni corection? All is on same dependent variable.
1
-
$\begingroup$ You might want to read this Q&A stats.stackexchange.com/questions/120362/… for some alternatives. $\endgroup$– mdeweyCommented Dec 18, 2016 at 11:49
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
The key point about the multiple testing problem is that every time you have a p-value you have to pay -- like rolling the dice in a casino. A table of p-values needs to be adjusted by the number of tests. Bonferroni is very conservative (i.e., will really lower the adjusted p-values) so you may want to invoke the false discovery rate (FDR) instead -- look up the 1995 paper by Benjamini & Hochberg. But yes, for 2-way and multiway ANOVA, most labs apply a Bonferroni correction.
-
$\begingroup$ There is also bootstrap and permutation p-value adjustment methods. This is covered in the book by Westfall and Young. "Resampling-Based Multiple Testing Examples and Methods for p-value Adjustment" Peter H. Westfall and S. Stanley Young, Wiley 1993. $\endgroup$ Commented Dec 18, 2016 at 1:31
-
$\begingroup$ Here is a URL for it. www.wiley.com/WileyCDA/WileyTitle/productCd-0471557617.html $\endgroup$ Commented Dec 18, 2016 at 1:34
-
$\begingroup$ The Westfall & Young procedure is also a good option. $\endgroup$– user32398Commented Dec 18, 2016 at 1:46
-
$\begingroup$ I noticed that Peter Westfall is also a member on CV. The advantage of the bootstrap approach is that it is not nearly as conservative as Bonferroni. I am pretty sure it deals with family-wise error rate and not False Discovery Rate. $\endgroup$ Commented Dec 18, 2016 at 2:10