When conducting a t-test why would one prefer to assume (or test for) equal variances rather than always use a Welch approximation of the df?

Question

It seems like when the assumption of homogeneity of variance is met that the results from a Welch adjusted t-test and a standard t-test are approximately the same. Why not simply always use the Welch adjusted t?

Answer 1

I would like to oppose the other two answers based on a paper (in German) by Kubinger, Rasch and Moder (2009).

They argue, based on "extensive" simulations from distributions either meeting or not meeting the assumptions imposed by a t-test, (normality and homogenity of variance) that the welch-tests performs equally well when the assumptions are met (i.e., basically same probability of committing alpha and beta errors) but outperforms the t-test if the assumptions are not met, especially in terms of power. Therefore, they recommend to always use the welch-test if the sample size exceeds 30.

As a meta-comment: For people interested in statistics (like me and probably most other here) an argument based on data (as mine) should at least count equally as arguments solely based on theoretical grounds (as the others here).

---

Update:
After thinking about this topic again, I found two further recommendations of which the newer one assists my point. Look at the original papers (which are both, at least for me, freely available) for the argumentations that lead to these recommendations.

The first recommendation comes from Graeme D. Ruxton in 2006: "If you want to compare the central tendency of 2 populations based on samples of unrelated data, then the unequal variance t-test should always be used in preference to the Student's t-test or Mann–Whitney U test."
In:
Ruxton, G.D., 2006. The unequal variance t-test is an underused alternative to Student’s t-test and the Mann–Whitney U test. Behav. Ecol. 17, 688–690.

The second (older) recommendation is from Coombs et al. (1996, p. 148): "In summary, the independent samples t test is generally acceptable in terms of controlling Type I error rates provided there are sufficiently large equal-sized samples, even when the equal population variance assumption is violated. For unequal-sized samples, however, an alternative that does not assume equal population variances is preferable. Use the James second-order test when distributions are either short-tailed symmetric or normal. Promising alternatives include the Wilcox H and Yuen trimmed means tests, which provide broader control of Type I error rates than either the Welch test or the James test and have greater power when data are long-tailed." (emphasis added)
In:
Coombs WT, Algina J, Oltman D. 1996. Univariate and multivariate omnibus hypothesis tests selected to control type I error rates when population variances are not necessarily equal. Rev Educ Res 66:137–79.

Answer 2

Of course, one could ditch both tests, and start using a Bayesian t-test (Savage-Dickey ratio test), which can account for equal and unequal variances, and best of all, it allows for a quantification of evidence in favor of the null hypothesis (which means, no more of old "failure to reject" talk)

This test is very simple (and fast) to implement, and there is a paper that clearly explains to readers unfamiliar with Bayesian statistics how to use it, along with an R script. You basically can just insert your data send the commands to the R console:

Wetzels, R., Raaijmakers, J. G. W., Jakab, E., & Wagenmakers, E.-J. (2009). How to Quantify Support For and Against the Null Hypothesis: A Flexible WinBUGS Implementation of a Default Bayesian t-test.

there is also a tutorial for all this, with example data:

http://www.ruudwetzels.com/index.php?src=SDtest

I know this is not a direct response to what was asked, but I thought readers might enjoy having this nice alternative

Answer 3

Because exact results are preferable to approximations, and avoid odd edge cases where the approximation may lead to a different result than the exact method.

The Welch method isn't a quicker way to do any old t-test, it's a tractable approximation to an otherwise very hard problem: how to construct a t-test under unequal variances. The equal-variance case is well-understood, simple, and exact, and therefore should always be used when possible.

Answer 4

Two reasons I can think of:

1. Regular Student's T is pretty robust to heteroscedasticity if the sample sizes are equal.

2. If you believe strongly a priori that the data is homoscedastic, then you lose nothing and might gain a small amount of power by using Studen'ts T instead of Welch's T.

One reason that I would not give is that Student's T is exact and Welch's T isn't. IMHO the exactness of Student's T is academic because it's only exact for normally distributed data, and no real data is exactly normally distributed. I can't think of a single quantity that people actually measure and analyze statistically where the distribution could plausibly have a support of all real numbers. For example, there are only so many atoms in the universe, and some quantities can't be negative. Therefore, when you use any kind of T-test on real data, you're making an approximation anyhow.

Answer 5

The fact that something more complex reduces to something less complex when some assumption is checked is not enough to throw the simpler method away.

Answer 6

With the assumption of equal variance, one can derive the non asymptotic distribution of t statistics. But when the assumption is violated, two variance terms cannot be cancelled, and we cannot simplify the distribution of statistics to a fixed one. Thus test can't be done.

Welch t test is an approximation which is robust and give an approximate degree of freedom. But it is not "exact", which means its type one error is not exactly what you want theoretically.

From my perspective, even when homogeneity test doesn't reject "equal variance", there is still risk to use t test assuming same variance. Because the true variance difference may be small/not significant, but not zero. We need a test without relying on "same variance", rather than use homogeneity test for "same variance".

https://arxiv.org/abs/2210.16473. Here is my new paper, hope it can help with your question. It can derive "exact" or non-asymptotic t statistics when variances of two groups are different, and it reaches the maximal degree of freedom that an "exact test" can allow. In small sample cases, it significantly outperforms Welch's t-test, in the sense of type one error. ($\mu_1=\mu_2=0$,$\sigma_1=1,\sigma_2=2,n_1=5,n_2=50$)

Its idea is: paired t test can always give an exact test, even when variances are unequal. But it loses some information when n1<n2, because it can only use n1 data points from n2 samples. This paper uses an orthogonal matrix to project the longer vector (n2) into a length (n1) vector, then we can use paired t test, with enough/compressed information.

Go here for instruction of use, a package developed by me: https://github.com/hobbitish1028/Te_test

Answer 7

I would take the opposite view here. Why bother with the Welch test when the standard unpaired student t test gives you nearly identical results. I studied this issue a while back and I explored a range of scenarios in an attempt to break down the t test and favor the Welch test. To do so I used sample sizes up to 5 times greater for one group vs the other. And, I explored variances up to 25 times greater for one group vs the other. And, it really did not make any material difference. The unpaired t test still generated a range of p values that were nearly identical to the Welch test.

You can see my work at the following link and focus especially on slide 5 and 6.

http://www.slideshare.net/gaetanlion/unpaired-t-test-family

Answer 8

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I'm usually reluctant to recommend the Welch correction because it's use is often deceptive. Which is, admittedly not a critique of the test itself.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.