2
$\begingroup$

If I have 3 groups that I want to compare with a t-test (correcting for multiple comparisons afterwards) but one of the groups has different variance than the other two, should I use a Welch correction for all comparisons or just for the comparison involving the group with unequal variance?

$\endgroup$
  • $\begingroup$ If you do an omnibus test first on all three means (ANOVA), you don't need to adjust for multiple comparisons in the specific case of 3 groups. This is called a protected Fisher's LSD, and the Type I error rate is maintained at .05 without further adjustment. $\endgroup$ – Patrick Coulombe Aug 5 '14 at 17:35
  • $\begingroup$ There's little loss in doing a Welch if the variances are equal or nearly equal. $\endgroup$ – Glen_b Aug 6 '14 at 3:35
1
$\begingroup$

You don't say how you plan to correct for multiple comparisons, but the implication ("afterwards") is that you will use a method based on p-values such as False Discovery Rate or others. If that is the case then it does not matter which t-test you use as long as you are comfortable with the assumptions going into the tests. So you can do the Welch test on all of them to be consistent, or if you prefer you can use the pooled test on the 2 that you believe to have the same variance. You could even combine a p-value from an ANOVA, another from a regression, one from a test of proportions, etc.

Why not try it both ways and see how they compare, if all the assumptions hold, then you should not see much of a difference.

You could further prove this to yourself (or find a counter-example if not true) by simulating the process and trying both ways, running it many times and seeing how the 2 methods compare.

$\endgroup$
1
$\begingroup$

Since oneway ANOVA assumes equal variances between each group, it would seem that your data violate this assumption. Perhaps a nonparametric omnibus test such as the Kruskal-Wallis test, followed by Dunn's test (with whatever family-wise error rate or false discovery rate adjustment for multiple comparisons you deem appropriate) would be the best course.

$\endgroup$
  • 1
    $\begingroup$ There's a Welch-Satterthwaite adjustment for ANOVA, just as with the t-test. Some packages offer it, I believe. I can't say that I recall having used it. $\endgroup$ – Glen_b Aug 6 '14 at 3:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.