I am comparing two groups (young vs. old adults) for differences in 5 brain regions (size) with independent t-tests. I also compare performance on three different aspects of a task with independent t-tests and then correlate brain sizes and behavioral performance . I adjusted my p-values (Bonferroni-method) for the correlations (for the five brain regions), but I am not sure, how exactly/if I have to adjust the p-values for each independent t-test between the age groups. I have a priori hypotheses about all comparisons. Any help would be highly appreciated!

  • $\begingroup$ Welcome to Cross Validated, Isabel. Before asking how to correct for multiple comparisons, you might consider whether you should correct. See here for some discussion: stats.stackexchange.com/questions/3200/… $\endgroup$ – Michael Lew Nov 21 '18 at 20:08

If your study is preliminary, or if it is about characterisation rather than testing hypotheses then you probably should not `correct' for multiple comparisons. The raw P-values give you the strength of evidence against the individual null hypotheses and are should be presented alongside a more detailed presentation of the data.

If you decide that you should `correct' then you will have to decide whether to treat all of your comparisons as part of the family or whether your experiments consist of a couple of families. I would suppose that the brain size tests are one family and the behavioural tests are another.

Finally, you might consider a slightly less power-wasting form of multiplicity correction like one of the step-down procedures (search for Šidák and step-down).


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.