17
$\begingroup$

The SPSS t-Test procedure reports 2 analyses when comparing 2 independent means, one analysis with equal variances assumed and one with equal variances not assumed. The degrees of freedom (df) when equal variances are assumed are always integer values (and equal n-2). The df when equal variances are not assumed are non-integer (e.g., 11.467) and nowhere near n-2. I am seeking an explanation of the logic and method used to calculate these non-integer df's.

$\endgroup$
  • 3
    $\begingroup$ A University of Florida PowerPoint presentation contains a good account of how this approximation to the sampling distribution of the Student t statistic is derived for the case of unequal variances. $\endgroup$ – whuber Sep 23 '14 at 22:35
  • $\begingroup$ Is the Welch's t-test always more accurate? Is there a downside to using the Welch approach? $\endgroup$ – Joel W. Sep 23 '14 at 23:47
  • $\begingroup$ If the Welch and the original t-test yield dramatically different p's, which should I go with? What if the p value for the differences in variances is only .06, but the differences in the p vales of the two t-tests are .000 and .121? (This occurred when one group of 2 had no variance and the other group of 25 had a variance of 70,000.) $\endgroup$ – Joel W. Sep 23 '14 at 23:49
  • 2
    $\begingroup$ Don't choose between them on the basis of the $p$ value. Unless you have good reason (before you even see the data) to assume equal variance, simply don't make that assumption. $\endgroup$ – Glen_b Sep 24 '14 at 0:58
  • 1
    $\begingroup$ The questions all relate to when to use the Welch test. This question has been posted at stats.stackexchange.com/questions/116610/… $\endgroup$ – Joel W. Sep 24 '14 at 15:23
13
$\begingroup$

The Welch-Satterthwaite d.f. can be shown to be a scaled weighted harmonic mean of the two degrees of freedom, with weights in proportion to the corresponding standard deviations.

The original expression reads:

$$\nu_{_W} = \frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2}{\frac{s_1^4}{n_1^2\nu_1}+\frac{s_2^4}{n_2^2\nu_2}}$$

Note that $r_i=s_i^2/n_i$ is the estimated variance of the $i^\text{th}$ sample mean or the square of the $i$-th standard error of the mean. Let $r=r_1/r_2$ (the ratio of the estimated variances of the sample means), so

\begin{align} \nu_{_W} &= \frac{\left(r_1+r_2\right)^2}{\frac{r_1^2}{\nu_1}+\frac{r_2^2}{\nu_2}} \newline \newline &=\frac{\left(r_1+r_2\right)^2}{r_1^2+r_2^2}\frac{r_1^2+r_2^2}{\frac{r_1^2}{\nu_1}+\frac{r_2^2}{\nu_2}} \newline \newline &=\frac{\left(r+1\right)^2}{r^2+1}\frac{r_1^2+r_2^2}{\frac{r_1^2}{\nu_1}+\frac{r_2^2}{\nu_2}} \end{align}

The first factor is $1+\text{sech}(\log(r))$, which increases from $1$ at $r=0$ to $2$ at $r=1$ and then decreases to $1$ at $r=\infty$; it's symmetric in $\log r$.

The second factor is a weighted harmonic mean:

$$H(\underline{x})=\frac{\sum_{i=1}^n w_i }{ \sum_{i=1}^n \frac{w_i}{x_i}}\,.$$

of the d.f., where $w_i=r_i^2$ are the relative weights to the two d.f.

Which is to say, when $r_1/r_2$ is very large, it converges to $\nu_1$. When $r_1/r_2$ is very close to $0$ it converges to $\nu_2$. When $r_1=r_2$ you get twice the harmonic mean of the d.f., and when $s_1^2=s_2^2$ you get the usual equal-variance t-test d.f., which is also the maximum possible value for $\nu_{_W}$ (given the sample sizes).

--

With an equal-variance t-test, if the assumptions hold, the square of the denominator is a constant times a chi-square random variate.

The square of the denominator of the Welch t-test isn't (a constant times) a chi-square; however, it's often not too bad an approximation. A relevant discussion can be found here.

A more textbook-style derivation can be found here.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Great insight about the harmonic mean, which is more appropriate than arithmetic mean for averaging ratios. $\endgroup$ – Felipe G. Nievinski Sep 1 '16 at 3:25
11
$\begingroup$

What you are referring to is the Welch-Satterthwaite correction to the degrees of freedom. The $t$-test when the WS correction is applied is often called Welch's $t$-test. (Incidentally, this has nothing to do with SPSS, all statistical software will be able to conduct Welch's $t$-test, they just don't usually report both side by side by default, so you wouldn't necessarily be prompted to think about the issue.) The equation for the correction is very ugly, but can be seen on the Wikipedia page; unless you are very math savvy or a glutton for punishment, I don't recommend trying to work through it to understand the idea. From a loose conceptual standpoint however, the idea is relatively straightforward: the regular $t$-test assumes the variances are equal in the two groups. If they're not, then the test should not benefit from that assumption. Since the power of the $t$-test can be seen as a function of the residual degrees of freedom, one way to adjust for this is to 'shrink' the df somewhat. The appropriate df must be somewhere between the full df and the df of the smaller group. (As @Glen_b notes below, it depends on the relative sizes of $s^2_1/n_1$ vs $s_2^2/n_2$; if the larger n is associated with a sufficiently smaller variance, the combined df can be lower than the larger of the two df.) The WS correction finds the right proportion of way from the former to the latter to adjust the df. Then the test statistic is assessed against a $t$-distribution with that df.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ For one t-test, SPSS reports the df as 26.608 but the n's for the two groups are 22 and 104. Are you sure about " The appropriate df must be somewhere between the full df and the df of the larger group"? (The standard deviations are 10.5 and 8.1 for the smaller and larger groups, respectively.) $\endgroup$ – Joel W. Sep 23 '14 at 22:08
  • 2
    $\begingroup$ It depends on the relative sizes of $s_1^2/n_1$ vs $s_2^2/n_2$. If the larger $n$ is associated with a sufficiently larger variance, the combined d.f. can be lower than the larger of the two d.f. Note that the Welch t-test is only approximate, since the squared denominator is not actually a (scaled) chi-square random variate. However in practice it does quite well. $\endgroup$ – Glen_b Sep 24 '14 at 0:55
  • $\begingroup$ I think I'll expand on the relationship between the relative sizes of the $(s_i^2/n_i)$ and the Welch d.f. in an answer (since it won't fit in a comment). $\endgroup$ – Glen_b Sep 24 '14 at 2:22
  • 1
    $\begingroup$ @Glen_b, I'm sure that will be of great value here. $\endgroup$ – gung - Reinstate Monica Sep 24 '14 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.