3
$\begingroup$

I understand the difference between a paired and an unpaired t-test but when should I use the students t-test and when the Welch's t-test?

$\endgroup$
2
  • $\begingroup$ If you say the difference (type it in a comment) between the two tests, I think you will answer your own question. $\endgroup$
    – Dave
    Jun 12, 2020 at 13:56
  • $\begingroup$ @Dave we are just thought to use them in R so what the difference is, is what I am trying to find out on my own now. Classes are not always that clear. $\endgroup$ Jun 12, 2020 at 13:59

1 Answer 1

6
$\begingroup$

The Welch t-test assumes that the variance between the two groups is different, so ostensibly you should use this test when you believe the two groups have different variances.

But, in my opinion the Welch t-test should be the default. Using the Welch t-test when the variances are actually the same results in smaller power. How much smaller? Well, it depends, but in my experience it isn't that much smaller. I'm a little busy at the moment but I will return with some simulations.

EDIT:

I ran some simple simulations for differences in means up to 4.0 with standard deviations ranging from 0.5 to 4.0. When the variances are the same, the welch t can have up to 5% smaller power. When the variances are different, Welch's t can have 3% more power.

$\endgroup$
8
  • $\begingroup$ So if two groups have, let's say, have a different standard deviation (variance squared), I should use the Welch t-test? $\endgroup$ Jun 12, 2020 at 14:08
  • $\begingroup$ His point, with which I agree, is that you should use the Welch test unless you have a very good reason to believe that you shouldn’t (either that you, somehow, know the variances to be equal or at least sufficiently close that Welch results in worse performance). Demetri, perhaps this warrants a separate question, but do you know if Welch’s original work proves any criteria or does simulations to show when his method beats equal-variance t-testing? $\endgroup$
    – Dave
    Jun 12, 2020 at 14:13
  • $\begingroup$ @Dave I'm not aware of Welch's original work provides any criteria of the kind you mention. I can create some toy simulations later today, but I'm in meetings until lunch time. $\endgroup$ Jun 12, 2020 at 14:14
  • $\begingroup$ @RuudVerhoef Use the Welch's T Test unless you have good reason not to (i.e. you have good reason to believe variances or standard devs are the same) $\endgroup$ Jun 12, 2020 at 14:15
  • $\begingroup$ Some recommending performing a test for equal variances to decide which to use (e.g., Levene's test, which is automatically printed by SPSS). I've done some simulations that showed it's preferable to just use Welch's test than to use Levene's test to decide which test to use. $\endgroup$
    – Noah
    Jun 12, 2020 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.