18
$\begingroup$

On this psychometrics website I read that

[A]t a deep level variance is a more fundamental concept than the standard deviation.

The site doesn't really explain further why variance is meant to be more fundamental than standard deviation, but it reminded me that I've read some similar things on this site.

For instance, in this comment @kjetil-b-halvorsen writes that "standard deviation is good for interpretation, reporting. For developing the theory the variance is better".

I sense that these claims are linked, but I don't really understand them. I understand that the square root of the sample variance isn't an unbiased estimator of the population standard deviation, but surely there must be more to it than that.

Maybe the term "fundamental" is too vague for this site. In that case, perhaps we can operationalize my question as asking whether variance is more important than standard deviation from the viewpoint of developing statistical theory. Why/why not?

Improve this question
$\endgroup$
2
  • $\begingroup$ Aren't they just the same thing? It's like 1+1 is the same as 2*1? $\endgroup$
    – SmallChess
    Commented Jun 9, 2016 at 3:43
  • 2
    $\begingroup$ The variance is the second cumulant, $\kappa_2$. The Wikipedia article on cumulants should impress anyone with how natural and important they are, not only for the study of random variables but also in physics and combinatorics. The multilinearity property (which is fundamental to performing calculations), as well as the extension of cumulants to multivariate distributions, are not enjoyed by the standard deviation. $\endgroup$
    – whuber ♦
    Commented Jun 9, 2016 at 15:07

4 Answers 4

Reset to default
16
$\begingroup$

Robert's and Bey's answers do give part of the story (i.e. moments tend to be regarded as basic properties of distributions, and conventionally standard deviation is defined in terms of the second central moment rather than the other way around), but the extent to which those things are really fundamental depends partly on what we mean by the term.

There would be no insurmountable problem, for example, if our conventions went the other way -- there's nothing stopping us conventionally defining some other sequence of quantities in place of the usual moments, say $E[(X-\mu)^p]^{1/p}$ for $p=1,2,3,...$ (note that $\mu$ fits into both the moment sequence and this one as the first term) and then defining moments -- and all manner of calculations in relation to moments -- in terms of them. Note that these quantities are all measured in the original units, which is one advantage over moments (which are in $p$-th powers of the original units, and so harder to interpret). This would make the population standard deviation the defined quantity and variance defined in terms of it.

However, it would make quantities like the moment generating function (or some equivalent relating to the new quantities defined above) rather less "natural", which would make things a little more awkward (but some conventions are a bit like that). There's some convenient properties of the MGF that would not be as convenient cast the other way.

More basic, to my mind (but related to it), is that there are a number of basic properties of variance that are more convenient when written as properties of variance than when written as properties of standard deviation (e.g. the variance of sums of independent random variables is the sum of the variances).

This additivity is a property that's not shared by other measures of dispersion and it has a number of important consequences.

[There are similar relationships between the other cumulants, so this is a sense in which we might want to define things in relation to moments more generally.]

All of these reasons are arguably either convention or convenience but to some extent it's a matter of viewpoint (e.g. from some points of view moments are pretty important quantities, from others they're not all that important). It may be that the "at a deep level" bit is intended to imply nothing more than kjetil's "when developing the theory".

I would agree with kjetil's point that you raised in your question; to some extent this answer is merely a hand-wavy discussion of it.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I would say that the two are in equal stead, each with their own set of accompanying conveniences. $\endgroup$
    – J. M. is not a statistician
    Commented Jun 9, 2016 at 7:16
2
$\begingroup$

Variance is defined by the first and second moments of a distribution. In contrast, the standard deviation is more like a "norm" than a moment. Moments are fundamental properties of a distribution, whereas norms are just ways to make a distinction.

Improve this answer
$\endgroup$
0
2
$\begingroup$

The variance is more fundamental than the standard deviation because the standard deviation is defined as 'the square root of the variance', e.g. its definition depends completely on the variance.

Variance, on the other hand is defined - completely independently - as the 'the expectation of the squared difference between a sample and the mean'.

Improve this answer
$\endgroup$
3
  • 3
    $\begingroup$ I'd see this more as a report on ways in which we (often) use terms, for example in teaching, not as a reflection on what is fundamental. It's perfectly possible to introduce standard deviation without mentioning variance (yet) and many texts and courses do precisely that, just as you can talk about Pythagoras' theorem without needing to use any special names for the squared quantities. Historically, the term variance in its statistical sense postdates that of standard deviation, so even this form of words was impossible for a few decades. $\endgroup$
    – Nick Cox
    Commented Jun 9, 2016 at 8:07
  • $\begingroup$ I became aware of the standard deviation having arisen as a label before the variance while trying to formulate a response to Glen's now deleted comment - at the time I reflected that the fact that the older term was now commonly defined in terms of the newer term strengthened the newer term's claims of being more fundamental rather than weakened them. $\endgroup$
    – Robert de Graaf
    Commented Jun 9, 2016 at 8:42
  • 1
    $\begingroup$ All kinds of explanations can be found. In my introductory teaching of SD (to geographers, not all of whom are strong mathematically), I don't use the term variance at all. I am quick to point out that SD is a natural scale measure for normal (Gaussian) distributions, as the distance between the mean and either inflection on the density function. I suspect that's more for my own amusement and pleasure than the students'. $\endgroup$
    – Nick Cox
    Commented Jun 9, 2016 at 8:57
0
$\begingroup$

In addition to the answers given here, one could point out that variance is more 'fundamental' than standard deviation in some sense, if we consider estimation from a (e.g. normal) population. For a sample of size $n$ drawn from a population $X$ with $\mathrm{Var}[X] = \sigma^2$, it is known that the sample variance $S^2$ is an unbiased estimator of $\sigma^2$, but $S$ is in general not an unbiased estimator of $\sigma$: $$ \mathrm{E}[S^2] = \sigma^2\,,\ \mathrm{E}[S]\neq \sigma\,, $$ see here, which follows from Jensen's inequality.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ On the contrary, the sample variance from a sample of size $n$ is not an unbiased estimator unless it is defined as having divisor $n - 1$. You can do that, but then the whole argument approaches circularity, that an estimator defined to be unbiased is indeed unbiased.. In addition, the argument here is contentious in assuming that being unbiased is absolutely preferable to being biased. Using maximum likelihood, for example, which many would regard as a deeper, more general principle that using unbiased estimators, often leads to biased estimators. $\endgroup$
    – Nick Cox
    Commented Jun 9, 2016 at 8:02
  • $\begingroup$ True, true. Unbiased is not always better, I never wanted to suggest otherwise. For the record, I tend to agree with everyone here that there would be no mathematical objections to working with standard dev. as fundamental concept instead of variance. Beauty is in the eye of the beholder. But what about glen_b 's remark about the useful properties of $\mathrm{Var}[]$, for example that $\mathrm{Var}[\sum_i X_i] = \sum_i \mathrm{Var}[X_i]$ for independent $X_i$? That seems to be pretty "natural". $\endgroup$
    – StijnDeVuyst
    Commented Jun 9, 2016 at 9:36
  • 1
    $\begingroup$ Indeed, the additivity of independent variances is a fundamental property, but that's not your argument. $\endgroup$
    – Nick Cox
    Commented Jun 9, 2016 at 9:39
  • $\begingroup$ Perhaps what's interesting is that, as with the mean, you can construct an unbiased estimator of variance without specifying a particular distribution (unbiased estimates of the standard deviation are distribution-specific.) $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Jul 22, 2016 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.