1
$\begingroup$

Suppose I want to compare groups A and B on 5 variables: age, education level, blood pressure, brain volume, and glucose level. Given that the assumptions are met, I will use the 2 sample t-test to make 5 comparisons. In this case, do I have to correct for multiple testing? What if I wanted to compare the groups on 10 variables? Or 20? Under what condition should I correct for multiple testing?

$\endgroup$
  • 1
    $\begingroup$ If you look through previous posts with the same tags you will find many answers to your question. Perhaps start withthis one $\endgroup$ – mdewey Aug 13 '16 at 16:40
  • $\begingroup$ Is your purpose confirmatory or exploratory? Do you need to publish the results or are you seeking solid information? $\endgroup$ – whuber Aug 13 '16 at 18:00
1
$\begingroup$

It's ultimately a matter of taste. How conservative do you want to be? What kind of error rates are you after—are you okay with a type-I error rate of .05 for each comparison, or do you want a rate of .05 for the whole set of comparisons, or something else?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.