31
$\begingroup$

Sorry if this is has been answered elsewhere, I haven't been able to find it.

I am wondering why we take the square root, in particular, of variance to create the standard deviation? What is it about taking the square root that produces a useful value?

$\endgroup$
2
  • $\begingroup$ Closely related: stats.stackexchange.com/questions/35123/… $\endgroup$
    – Sycorax
    Mar 23, 2017 at 18:20
  • 2
    $\begingroup$ Think about the standard deviation as a euclidean vector norm and then the variance as the square. This definition of variance and standard deviation turn out to have useful analytical properties. $\endgroup$ Mar 23, 2017 at 22:28

5 Answers 5

52
$\begingroup$

In some sense this is a trivial question, but in another, it is actually quite deep!

  • As others have mentioned, taking the square root implies $\operatorname{Stdev}(X)$ has the same units as $X$.

  • Taking the square root gives you absolute homogeneity aka absolute scalability. For any scalar $\alpha$ and random variable $X$, we have: $$ \operatorname{Stdev}[\alpha X] = |\alpha| \operatorname{Stdev}[X]$$ Absolute homogeneity is a required property of a norm. The standard deviation can be interpreted as a norm (on the vector space of mean zero random variables) in a similar way that $\sqrt{x^2 + y^2+z^2}$ is the standard Euclidian norm in a three-dimensional space. The standard deviation is a measure of distance between a random variable and its mean.

Standard deviation and the $L_2$ norm

Finite dimension case:

In an $n$ dimensional vector space, the standard Euclidian norm aka the $L_2$ norm is defined as:

$$\|\mathbf{x}\|_2 = \sqrt{\sum_i x_i^2}$$

More broadly, the $p$-norm $\|\mathbf{x}\|_p = \left(\sum_i |x_i|^p \right)^{\frac{1}{p}}$ takes the $p$th root to get absolute homogeneity: $\|\alpha \mathbf{x}\|_p = \left( \sum_i |\alpha x_i|^p \right)^\frac{1}{p} = | \alpha | \left( \sum_i |x_i|^p \right)^\frac{1}{p} = |\alpha | \|\mathbf{x}\|_p $.

If you have weights $q_i$ then the weighted sum $\sqrt{\sum_i x_i^2 q_i}$ is also a valid norm. Furthermore, it's the standard deviation if $q_i$ represent probabilities and $\operatorname{E}[\mathbf{x}] \equiv \sum_i x_i q_i = 0$

Infinite dimension case:

In an infinite dimensional Hilbert Space we similarly may define the $L_2$ norm:

$$ \|X\|_2 = \sqrt{\int_\omega X(\omega)^2 dP(\omega) }$$

If $X$ is a mean zero random variable and $P$ is the probability measure, what's the standard deviation? It's the same: $\sqrt{\int_\omega X(\omega)^2 dP(\omega) }$.

Summary:

Taking the square root makes means the standard deviation satisfies absolute homogeneity, a required property of a norm.

On a space of random variables, $\langle X, Y \rangle = \operatorname{E}[XY]$ is an inner product and $\|X\|_2 = \sqrt{\operatorname{E}[X^2]}$ the norm induced by that inner product. Thus the standard deviation is the norm of a demeaned random variable: $$\operatorname{Stdev}[X] = \|X - \operatorname{E}[X]\|_2$$ It's a measure of distance from mean $\operatorname{E}[X]$ to $X$.

(Technical point: while $\sqrt{\operatorname{E}[X^2]}$ is a norm, the standard deviation $\sqrt{\operatorname{E}[(X - \operatorname{E}[X])^2]}$ isn't a norm over random variables in general because a requirement for a normed vector space is $\|x\| = \mathbf{0}$ if and only if $x = \mathbf{0}$. A standard deviation of 0 doesn't imply the random variable is the zero element.)

$\endgroup$
1
  • 2
    $\begingroup$ This answer really gets at the heart of the issue, making it more informative than the currently accepted one. $\endgroup$ Mar 28, 2017 at 16:32
27
$\begingroup$

Variance of $X$ is defined as $V(X) = E(X-E(X))^2$, so it is an expectation of a squared difference between X and its expected value.

If $X$ is time in seconds, $X-E(X)$ is in seconds, but $V(X)$ is in $\mbox{seconds}^2$ and $\sqrt{V(X)}$ is again in seconds.

$\endgroup$
6
  • $\begingroup$ Ah I see, it's just undoing the change in scale that resulted from squaring the differences, in the calculation of variance? $\endgroup$
    – Dave
    Mar 23, 2017 at 18:33
  • 11
    $\begingroup$ Right – but change in dimensions, not in scale. $\endgroup$ Mar 24, 2017 at 8:17
  • $\begingroup$ But it's not like there is a single term there: there are many and each when in power 2, brings more or less than other terms. But when we take the square root, we kind of neglect of that difference, don't we? We wouldn't get the initial numerator, sum of all the differences that way. Wouldn't it be better to take a square root of each individual term? $\endgroup$
    – parsecer
    Mar 24, 2017 at 19:36
  • $\begingroup$ It sounds like you are thinking about the estimate $\hat{V}$, based on a sample. In that case, if you did so, the differences would zero out: $\sum_{i=1}^n (x_i - \bar{x}) = \sum_{i=1}^n x_i - \sum_{i=1}^n x_i = 0$. $\endgroup$
    – HStamper
    Mar 24, 2017 at 20:22
  • $\begingroup$ @EricMittman Except that $\sqrt{a^2} = \lvert a \rvert$, not $a$, in which case you'd get the mean absolute error. $\endgroup$
    – Danica
    Mar 25, 2017 at 13:20
6
$\begingroup$

The simple answer is that the units are on the same scale as the mean. Example: I estimate the mean for secondary student to be 160cm with a standard deviation (SD) of 20cm. It is intuitively easier to get a sense of the variation with the SD than the variance of 400cm^2.

$\endgroup$
0
0
$\begingroup$

In more simple terms standard deviation is designed to give us a positive number that says something about the spread of our data about it's mean.

If we were to just add up the distances of all the points from the mean, then points in the positive and negative directions would combine in a way that would tend to gravitate back toward the mean and we would lose information about the spread. This is why we measure variance first, so that all of the distances are preserved as positive quantities via squaring and they don't cancel each other out. In the end we want a positive value that represents the units we started with - this has already been commented on above - so we take the positive square root.

$\endgroup$
-5
$\begingroup$

It is a historical stupidity which we continue due to intellectual laziness. They chose to square the differences from the mean in order to get rid of the minus sign. Then they took the square root so as to bring it to a scale similar to the mean.

Someone should generate new statistics, computing variance and SD using modulus or absolute values of deviance from the mean. This would get rid of this whole squaring and then taking the square root business.

$\endgroup$
4
  • 2
    $\begingroup$ We have that already, in the form of the mean (or median) absolute deviation, L1 norms, and the like. However, the major advantage of the traditional approach is that, unlike absolute values, it's differentiable, which allows you to analytically minimize and maximize things. $\endgroup$ Jun 4, 2019 at 2:03
  • 2
    $\begingroup$ You fail to provide substantive justification for your stance, please provide a clearly laid out mathematical argument. The sum of absolute values scale very differently to the square root of the sum of squares. The latter emphasises the contribution of extreme values, which is a useful property. Also, SSQ is central to least squares analytic methods. Please take the time to expand on the problems of SD and how the alternatives compare so readers can understand your point of view. . $\endgroup$
    – ReneBt
    Jun 4, 2019 at 3:23
  • $\begingroup$ (-1) It's all too easy to read phrases like "historical stupidity" and "intellectual laziness" as being self-referential. $\endgroup$
    – whuber
    Jun 4, 2019 at 12:46
  • $\begingroup$ @MattKrause I agree with the main substance of your point about differentiability. I will mention just for a little extra nuance that sometimes weak derivatives exist that can lead to weak solutions to optimization problems. $\endgroup$
    – Galen
    Apr 14 at 20:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.