7
$\begingroup$

According to statistics textbooks, t-tests require the dependent variable to be normally distributed and the variance to be homogenous across conditions.

What's the authorized standard in statistics? Is it OK or even necessary to check homogeneity before running a t-test?

Improve this question
$\endgroup$
12
  • $\begingroup$ Someone can point me to a relevant citation if I am mistaken, but I have never heard of any controversy surrounding whether or not to check for homogeneity of variance before running a t-test (and the commenter over at SO does not provide any references). It seems like it would be a good idea to always check that assumptions are met when running statistical tests...? You can get around this altogether by always running a Welch's t-test, which does not assume homogeneity of variance. This is the default option in R, using t.test $\endgroup$
    – Mark White
    Commented Jul 8, 2017 at 1:03
  • 3
    $\begingroup$ @Mark Indeed -- a number of questions on site give several references that indicate that you're generally better off just using a Welch test than testing equality of variance in order to choose between an equal variance test and an unequal variance test. $\endgroup$
    – Glen_b
    Commented Jul 8, 2017 at 2:10
  • 1
    $\begingroup$ @WhiteGirl There's no "authorized standard"... which is largely a good thing, not a bad thing. Better to learn how to figure out how to choose between tests (e.g. by the simulations Mark mentions) and explain the reasoning (which can be situational) than to bow to some arbitrary standard which cannot hope to be suitable for every circumstance. (Considering the frequent bad practice I see caused by slavish adherence to decades-old rules of thumb, I'd really prefer not to encourage even more of it.) $\endgroup$
    – Glen_b
    Commented Jul 8, 2017 at 2:37
  • 1
    $\begingroup$ Note that a hypothesis test of variance answers the wrong question. It looks at whether the variances differ more than would be accounted for by random variation. The question we want to answer is not that one -- we would rarely believe that the population variances are exactly equal, for one thing, so in large samples it will tell us what we already know and in small samples it will fail to tell us even what we already know. What matters is the impact of the heteroskedasticity that we have on our inference about the means.... ctd $\endgroup$
    – Glen_b
    Commented Jul 8, 2017 at 2:42
  • 1
    $\begingroup$ ctd.. How different they might be is more important than whether we can tell that they're different (in small samples failure to reject would be no consolation, in large sample a rejection may barely matter -- because we may pick up even trivial differences). The test also can't tell us that if the sample sizes are equal, it hardly matters whether we assume equal variance. $\endgroup$
    – Glen_b
    Commented Jul 8, 2017 at 2:45

3 Answers 3

Reset to default
10
$\begingroup$

No, it is not necessary. Given that there is a test that accounts for heterogeneous variances (Welch's t-test), you can simply conduct it. For one, the tests for homogeneity of variance (HOV) are problematic in a number of ways. Some lack power, they - like other statistical tests - are too powerful with large sample sizes, effect sizes are missing for these tests, some are faulty under non-normality, ...

The typical approach for most applied researchers is to conduct Levene's test, then decide whether to conduct Student's t-test or Welch's t-test based on the result of Levene's test. However, Zimmerman (2004) showed through simulation that conditioning the test on the result of Levene's test distorts the p-value of the test i.e. your p-value from Student's or Welch's is not reliable when you choose which one to do based on Levene's test. Furthermore, given that Welch's test is almost as powerful as Student's test under HOV, and it is much more powerful when HOV is absent, it is advisable to "just do Welch's test".

Zimmerman, D. W. (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology, 57(1), 173–181. https://doi.org/10.1348/000711004849222

Here is another paper that gives the same basic advice: Delacre, M., Lakens, D., & Leys, C. (2017). Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test. International Review of Social Psychology, 30(1), 92–101. https://doi.org/10.5334/irsp.82

Improve this answer
$\endgroup$
4
  • 3
    $\begingroup$ Note that R uses Welch's t-test by default. $\endgroup$
    – Sal Mangiafico
    Commented Jul 8, 2017 at 12:33
  • 2
    $\begingroup$ Yeah, but most applied researchers do not use R. Many use SPSS and are taught to decide based on Levene's test. $\endgroup$
    – Heteroskedastic Jim
    Commented Jul 8, 2017 at 14:49
  • 1
    $\begingroup$ Agreed. When I learned on SAS, I believe we looked at a type of F-test on the output to decide which t-test output to use.... But the original poster tagged R. $\endgroup$
    – Sal Mangiafico
    Commented Jul 8, 2017 at 14:53
  • 2
    $\begingroup$ Oh, thanks! That's something else I've now learned, check user tags. I used to teach the same thing in SPSS, until I read the second article up there. Then I read the other article which one can find on the Wikipedia Welch t page. Then I ran some simulations, and I now teach "just do Welch". With small sample sizes though, it gets nuanced, but doesn't everything? $\endgroup$
    – Heteroskedastic Jim
    Commented Jul 8, 2017 at 14:59
1
$\begingroup$

According to statistics textbooks, t-tests require the dependent variable to be normally distributed and the variance to be homogenous across conditions

This is misleading. Generally, introductory statistics textbooks teach 2 (maybe 3 if you count paired stuff) two sample t-tests. Both tests assume that each of the two random samples are iid normal random samples, and are independent between each other. However they are different in that

  1. one assumes further that the two groups have equal variance,
  2. one makes no additional assumptions, but the sampling distribution of your test statistic is only approximately t-distributed.

The assumption that both groups have the same variance is unverifiable. This is because this is an assumption about unobservable variance parameters. However, 1) there do exist tests that can test equality of variances between the two groups, and 2) you can sometimes reassure yourself looking at, perhaps, histograms of the two sets of data, checking to make sure they have the same variance, roughly.

Regarding the first technique: like any hypothesis tests, there are the associated type 1 and type 2 error events. If you decide to formally test equality of variances before you test the means, since you are running two tests, you need to realize that there is some type 1 and type 2 error for your overall strategy.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Then, do we need variance homogeneity check before t.test? $\endgroup$
    – WhiteGirl
    Commented Jul 8, 2017 at 1:37
  • $\begingroup$ I mentioned two options; I would do atleast one of them. Comparing these two techniques would require more information about your specific problem. $\endgroup$
    – Taylor
    Commented Jul 8, 2017 at 1:40
  • $\begingroup$ The assumption that both groups have the same variance is unverifiable.??var.test and bartlett.test in R can do that! $\endgroup$
    – WhiteGirl
    Commented Jul 8, 2017 at 2:58
  • $\begingroup$ @WhiteGirl we are not sharing the definition of unverifiable. Please read the rest of my answer. $\endgroup$
    – Taylor
    Commented Jul 8, 2017 at 15:32
1
$\begingroup$

Not only is it not necessary, see user162986's answer, it can also imperil the interpretability of your test.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.