The guidance on this site concerning when to use the Welch t-test is somewhat general in nature (When conducting a t-test why would one prefer to assume (or test for) equal variances rather than always use a Welch approximation of the df? ). Are there specific, widely accepted rules of thumb such as related to: (1) Size of the difference in variances as opposed to statistical significance of the difference, (2) Sample size after which the robustness of the traditional t-test makes the Welch test unnecessary, (3) Other considerations that might best inform a decision on whether to use the Welch t-test, (4) What extent of benefit the Welch t-test might provide in various situations, and (5) Do the Welch and student t-test converge with large N? Any informed guidance would be greatly appreciated.

  • $\begingroup$ When you assume the two groups have equal variances (from same population or under $H_0$), I would think traditional t-test should be used. Or Welch t-test should be considered for the adjustment of degree of freedom. $\endgroup$ – David Z Sep 24 '14 at 15:30
  • $\begingroup$ The convergence depends largely on the $n_0$ and $n_1$ whether they are equal or not. The more the difference, the harder the convergence. $\endgroup$ – David Z Sep 24 '14 at 15:36
  • 3
    $\begingroup$ Both type I and type II error rates are impacted by choosing tests on the basis of tests of assumptions. A number of studies recommended not testing equality of variance before deciding whether to assume it is true. See here. You rarely lose much by using Welch by default. $\endgroup$ – Glen_b -Reinstate Monica Sep 24 '14 at 16:06
  • 1
    $\begingroup$ Your efforts to make the original question more specific, quantitative, and thorough are appreciated. However, it still seems that the accepted answer to the duplicate question renders (1) through (4) moot by citing evidence concluding you should just use the Welch test, period. (5) is so well known that I doubt any thread here actually states it explicitly, but you will find many statements in the form of "any (reasonable) hypothesis test becomes arbitrarily powerful as the sample size increases," which answers (5) positively. $\endgroup$ – whuber Sep 24 '14 at 21:39