26
$\begingroup$

Sorry if this is has been answered elsewhere, I haven't been able to find it.

I am wondering why we take the square root, in particular, of variance to create the standard deviation? What is it about taking the square root that produces a useful value?

$\endgroup$
  • $\begingroup$ Closely related: stats.stackexchange.com/questions/35123/… $\endgroup$ – Sycorax Mar 23 '17 at 18:20
  • 2
    $\begingroup$ Think about the standard deviation as a euclidean vector norm and then the variance as the square. This definition of variance and standard deviation turn out to have useful analytical properties. $\endgroup$ – theideasmith Mar 23 '17 at 22:28
44
$\begingroup$

In some sense this is a trivial question, but in another, it is actually quite deep!

  • As others have mentioned, taking the square root implies $\operatorname{Stdev}(X)$ has the same units as $X$.

  • Taking the square root gives you absolute homogeneity aka absolute scalability. For any scalar $\alpha$ and random variable $X$, we have: $$ \operatorname{Stdev}[\alpha X] = |\alpha| \operatorname{Stdev}[X]$$ Absolute homogeneity is a required property of a norm. The standard deviation can be interpreted as a norm (on the vector space of mean zero random variables) in a similar way that $\sqrt{x^2 + y^2+z^2}$ is the standard Euclidian norm in a three-dimensional space. The standard deviation is a measure of distance between a random variable and its mean.

Standard deviation and the $L_2$ norm

Finite dimension case:

In an $n$ dimensional vector space, the standard Euclidian norm aka the $L_2$ norm is defined as:

$$\|\mathbf{x}\|_2 = \sqrt{\sum_i x_i^2}$$

More broadly, the $p$-norm $\|\mathbf{x}\|_p = \left(\sum_i |x_i|^p \right)^{\frac{1}{p}}$ takes the $p$th root to get absolute homogeneity: $\|\alpha \mathbf{x}\|_p = \left( \sum_i |\alpha x_i|^p \right)^\frac{1}{p} = | \alpha | \left( \sum_i |x_i|^p \right)^\frac{1}{p} = |\alpha | \|\mathbf{x}\|_p $.

If you have weights $q_i$ then the weighted sum $\sqrt{\sum_i x_i^2 q_i}$ is also a valid norm. Furthermore, it's the standard deviation if $q_i$ represent probabilities and $\operatorname{E}[\mathbf{x}] \equiv \sum_i x_i q_i = 0$

Infinite dimension case:

In an infinite dimensional Hilbert Space we similarly may define the $L_2$ norm:

$$ \|X\|_2 = \sqrt{\int_\omega X(\omega)^2 dP(\omega) }$$

If $X$ is a mean zero random variable and $P$ is the probability measure, what's the standard deviation? It's the same: $\sqrt{\int_\omega X(\omega)^2 dP(\omega) }$.

Summary:

Taking the square root makes means the standard deviation satisfies absolute homogeneity, a required property of a norm.

On a space of random variables, $\langle X, Y \rangle = \operatorname{E}[XY]$ is an inner product and $\|X\|_2 = \sqrt{\operatorname{E}[X^2]}$ the norm induced by that inner product. Thus the standard deviation is the norm of a demeaned random variable: $$\operatorname{Stdev}[X] = \|X - \operatorname{E}[X]\|_2$$ It's a measure of distance from mean $\operatorname{E}[X]$ to $X$.

(Technical point: while $\sqrt{\operatorname{E}[X^2]}$ is a norm, the standard deviation $\sqrt{\operatorname{E}[(X - \operatorname{E}[X])^2]}$ isn't a norm over random variables in general because a requirement for a normed vector space is $\|x\| = \mathbf{0}$ if and only if $x = \mathbf{0}$. A standard deviation of 0 doesn't imply the random variable is the zero element.)

$\endgroup$
  • 1
    $\begingroup$ This answer really gets at the heart of the issue, making it more informative than the currently accepted one. $\endgroup$ – 00prometheus Mar 28 '17 at 16:32
26
$\begingroup$

Variance of $X$ is defined as $V(X) = E(X-E(X))^2$, so it is an expectation of a squared difference between X and its expected value.

If $X$ is time in seconds, $X-E(X)$ is in seconds, but $V(X)$ is in $\mbox{seconds}^2$ and $\sqrt{V(X)}$ is again in seconds.

$\endgroup$
  • $\begingroup$ Ah I see, it's just undoing the change in scale that resulted from squaring the differences, in the calculation of variance? $\endgroup$ – Dave Mar 23 '17 at 18:33
  • 11
    $\begingroup$ Right – but change in dimensions, not in scale. $\endgroup$ – Jean-François Corbett Mar 24 '17 at 8:17
  • $\begingroup$ But it's not like there is a single term there: there are many and each when in power 2, brings more or less than other terms. But when we take the square root, we kind of neglect of that difference, don't we? We wouldn't get the initial numerator, sum of all the differences that way. Wouldn't it be better to take a square root of each individual term? $\endgroup$ – parsecer Mar 24 '17 at 19:36
  • $\begingroup$ It sounds like you are thinking about the estimate $\hat{V}$, based on a sample. In that case, if you did so, the differences would zero out: $\sum_{i=1}^n (x_i - \bar{x}) = \sum_{i=1}^n x_i - \sum_{i=1}^n x_i = 0$. $\endgroup$ – HStamper Mar 24 '17 at 20:22
  • $\begingroup$ @EricMittman Except that $\sqrt{a^2} = \lvert a \rvert$, not $a$, in which case you'd get the mean absolute error. $\endgroup$ – Dougal Mar 25 '17 at 13:20
6
$\begingroup$

The simple answer is that the units are on the same scale as the mean. Example: I estimate the mean for secondary student to be 160cm with a standard deviation (SD) of 20cm. It is intuitively easier to get a sense of the variation with the SD than the variance of 400cm^2.

$\endgroup$
0
$\begingroup$

In more simple terms standard deviation is designed to give us a positive number that says something about the spread of our data about it's mean.

If we were to just add up the distances of all the points from the mean, then points in the positive and negative directions would combine in a way that would tend to gravitate back toward the mean and we would lose information about the spread. This is why we measure variance first, so that all of the distances are preserved as positive quantities via squaring and they don't cancel each other out. In the end we want a positive value that represents the units we started with - this has already been commented on above - so we take the positive square root.

$\endgroup$
-3
$\begingroup$

It is a historical stupidity which we continue due to intellectual laziness. They chose to square the differences from the mean in order to get rid of the minus sign. Then they took the square root so as to bring it to a scale similar to the mean.

Someone should generate new statistics, computing variance and SD using modulus or absolute values of deviance from the mean. This would get rid of this whole squaring and then taking the square root business.

$\endgroup$
  • 1
    $\begingroup$ We have that already, in the form of the mean (or median) absolute deviation, L1 norms, and the like. However, the major advantage of the traditional approach is that, unlike absolute values, it's differentiable, which allows you to analytically minimize and maximize things. $\endgroup$ – Matt Krause Jun 4 at 2:03
  • 1
    $\begingroup$ You fail to provide substantive justification for your stance, please provide a clearly laid out mathematical argument. The sum of absolute values scale very differently to the square root of the sum of squares. The latter emphasises the contribution of extreme values, which is a useful property. Also, SSQ is central to least squares analytic methods. Please take the time to expand on the problems of SD and how the alternatives compare so readers can understand your point of view. . $\endgroup$ – ReneBt Jun 4 at 3:23
  • $\begingroup$ (-1) It's all too easy to read phrases like "historical stupidity" and "intellectual laziness" as being self-referential. $\endgroup$ – whuber Jun 4 at 12:46

protected by whuber Jun 4 at 12:44

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.