6
$\begingroup$

Why do we calculate the pooled standard deviation by averaging the variances and taking the square root, rather than averaging the standard deviations directly?

Edit: this came up in the context of creating an effect size for a paired samples t-test, but if the answer varies across contexts I am interested to learn about that as well.

Improve this question
$\endgroup$
6
  • $\begingroup$ Welcome to CV. Your question is a bit sparse in explanation. Would it be possible for you to amplify or further elaborate what you mean? $\endgroup$
    – user78229
    Commented Mar 26, 2016 at 0:44
  • $\begingroup$ What context are you pooling in? $\endgroup$
    – Glen_b
    Commented Mar 26, 2016 at 1:30
  • 1
    $\begingroup$ Averaging the SDs directly would be the mathematical equivalent of claiming that the length of the hypotenuse of any right triangle is the sum of the lengths of its legs. Except in special cases, it would simply give the wrong result. $\endgroup$
    – whuber ♦
    Commented Mar 26, 2016 at 23:43
  • $\begingroup$ @whuber, I know they are not mathematically the same formula. Why is the pooled standard deviation created from averaging the variances more useful than averaging the standard deviations? Presumably the convention exists because this mathematical value is more useful? $\endgroup$
    – gabryll
    Commented Mar 27, 2016 at 17:16
  • 1
    $\begingroup$ You could just as well ask why, say, subtracting one date from another is more useful for finding an elapsed time than dividing the dates: the former has a meaning that the latter does not have. It's the same here: averaging SDs has little or no meaning when it comes to describing variation or uncertainty in data in any coherent, non-arbitrary, objective way. $\endgroup$
    – whuber ♦
    Commented Mar 28, 2016 at 17:11

1 Answer 1

Reset to default
6
$\begingroup$

We work with variances rather than standard deviations because variances have special properties.

In particular, variances of sums and differences of variables have a simple form, and if the variables are independent, the result is even simpler.

That is, if two variables are independent, the variance of the difference is the sum of the variances ("variances add" -- but standard deviations don't).

Specifically, in say a two-sample t test, we're trying to find the standard deviation of the difference in sample means. We can use basic properties of variance (linked above) to see that the variance of the individual sample means is $\sigma^2/n$, which we can estimate by $s^2/n$ for each sample.

Now that we have the variance of each the means, we can use the "variances add" result to get that the variance of the difference of the means is the sum of the two variances of the sample means. So the standard deviation of the distribution of the difference in means (the standard error of the difference in means) is the square root of that sum.

This works quite directly for the Welch t-test, where we estimate $\text{Var}(\bar{X}-\bar{Y})$ by $s_x^2/n_x+s_y^2/n_y$. The equal-variance version works using the same idea but because the variances are assumed identical, there we produce a single overall estimate of $\sigma^2$ from both samples. That is, we add together all the squared deviations from the corresponding group mean before dividing by the total d.f. from the two groups (each loses 1 d.f. because we measure deviations from the individual group means). This corresponds to a form of d.f.-weighted average of the individual variances $s^2_p=w_xs^2_x+w_ys^2_y$ where $w_x=\text{df}_x/(\text{df}_x+\text{df}_y)$. Then that single estimate of pooled variance $s^2_p$ is used in an estimate of the variance of the difference in means. Since $\text{Var}(\bar{X})=\sigma^2/n_x$ and $\text{Var}(\bar{Y})=\sigma^2/n_y$, again the variance of the sum is the sum of the variances, so $\text{Var}(\bar{X}-\bar{Y})=\sigma^2/n_x+\sigma^2/n_y$, which we again then estimate by replacing $\sigma^2$ by the estimate $s^2_p$.

In either case, we can standardize our difference in means by dividing by the corresponding estimate of standard error. In both cases this is where the denominator of the $t$-statistic comes from.

Similar results come up in other cases.

Improve this answer
$\endgroup$
4
  • $\begingroup$ If $\mathrm{cor}(X,Y) = 1$, then $\mathrm{sd}(X+Y) = \mathrm{sd}(X) + \mathrm{sd}(Y)$. In light of this, I think it's fair to say that just because it's true that variances add when $X, Y$ have correlation $0$, doesn't automatically mean it's why we do it. We have to explain why purely uncorrelated random variables are more common to come across (or more useful?) than perfectly correlated ones, I think. $\endgroup$
    – user795305
    Commented May 10, 2017 at 22:25
  • 1
    $\begingroup$ @Ben See my links to the Wikipedia page on basic properties of variance, which shows the non independence case. Our models are often independence models, but it's always true that $\text{Var}(aX+bY)=a^2\sigma^2_X+2ab\sigma_X\sigma_Y\rho_{X,Y}+b^2\sigma^2_Y$. There's no correspondingly-as-simple result for standard deviations (at least that's not simpler still by working in terms of variances), except when $\rho=\pm 1$. How often are we dealing with that case, compared to dealing with $\rho\neq \pm 1$? $\endgroup$
    – Glen_b
    Commented May 10, 2017 at 23:22
  • $\begingroup$ Yeah, I completely agree. Having $|\rho| = 1$ gives that the random variables are almost surely proportional. This case appears to be much, much less interesting. I'm just trying to point out that I think that an answer to the question should include this. My understanding of your answer is that (simply put and very roughly) "independence gives variances adding, and this is widely useful (seen via t test)" However, a similar argument can't be made for having $|\rho|=1$ since that case is uninteresting. The argument concludes by then noticing that neither the variance nor sd add in other cases. $\endgroup$
    – user795305
    Commented May 11, 2017 at 0:37
  • 1
    $\begingroup$ But the variance does something almost as simple as adding, which is why my second sentence is deliberately phrased precisely the way it is. I don't want to add something like what you say there because I disagree with the equivalence it draws between variance and standard deviation when $|\rho|$ is neither $0$ or $1$. Variance is simpler than standard deviation in those cases, and (outside some limited special cases) it's unique among all the ways of measuring spread-outed-ness in that simplicity. $\endgroup$
    – Glen_b
    Commented May 11, 2017 at 0:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.