Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Sorry if this is has been answered elsewhere, I haven't been able to find it.
I am wondering why we take the square root, in particular, of variance to create the standard deviation? What is it about taking the square root that produces a useful value?
In some sense this is a trivial question, but in another, it is actually quite deep!
As others have mentioned, taking the square root implies $\operatorname{Stdev}(X)$ has the same units as $X$.
Taking the square root gives you absolute homogeneity aka absolute scalability. For any scalar $\alpha$ and random variable $X$, we have: $$ \operatorname{Stdev}[\alpha X] = |\alpha| \operatorname{Stdev}[X]$$ Absolute homogeneity is a required property of a norm. The standard deviation can be interpreted as a norm (on the vector space of mean zero random variables) in a similar way that $\sqrt{x^2 + y^2+z^2}$ is the standard Euclidian norm in a three-dimensional space. The standard deviation is a measure of distance between a random variable and its mean.
In an $n$ dimensional vector space, the standard Euclidian norm aka the $L_2$ norm is defined as:
$$\|\mathbf{x}\|_2 = \sqrt{\sum_i x_i^2}$$
More broadly, the $p$-norm $\|\mathbf{x}\|_p = \left(\sum_i |x_i|^p \right)^{\frac{1}{p}}$ takes the $p$th root to get absolute homogeneity: $\|\alpha \mathbf{x}\|_p = \left( \sum_i |\alpha x_i|^p \right)^\frac{1}{p} = | \alpha | \left( \sum_i |x_i|^p \right)^\frac{1}{p} = |\alpha | \|\mathbf{x}\|_p $.
If you have weights $q_i$ then the weighted sum $\sqrt{\sum_i x_i^2 q_i}$ is also a valid norm. Furthermore, it's the standard deviation if $q_i$ represent probabilities and $\operatorname{E}[\mathbf{x}] \equiv \sum_i x_i q_i = 0$
In an infinite dimensional Hilbert Space we similarly may define the $L_2$ norm:
$$ \|X\|_2 = \sqrt{\int_\omega X(\omega)^2 dP(\omega) }$$
If $X$ is a mean zero random variable and $P$ is the probability measure, what's the standard deviation? It's the same: $\sqrt{\int_\omega X(\omega)^2 dP(\omega) }$.
Taking the square root makes means the standard deviation satisfies absolute homogeneity, a required property of a norm.
On a space of random variables, $\langle X, Y \rangle = \operatorname{E}[XY]$ is an inner product and $\|X\|_2 = \sqrt{\operatorname{E}[X^2]}$ the norm induced by that inner product. Thus the standard deviation is the norm of a demeaned random variable: $$\operatorname{Stdev}[X] = \|X - \operatorname{E}[X]\|_2$$ It's a measure of distance from mean $\operatorname{E}[X]$ to $X$.
(Technical point: while $\sqrt{\operatorname{E}[X^2]}$ is a norm, the standard deviation $\sqrt{\operatorname{E}[(X - \operatorname{E}[X])^2]}$ isn't a norm over random variables in general because a requirement for a normed vector space is $\|x\| = \mathbf{0}$ if and only if $x = \mathbf{0}$. A standard deviation of 0 doesn't imply the random variable is the zero element.)
Variance of $X$ is defined as $V(X) = E(X-E(X))^2$, so it is an expectation of a squared difference between X and its expected value.
If $X$ is time in seconds, $X-E(X)$ is in seconds, but $V(X)$ is in $\mbox{seconds}^2$ and $\sqrt{V(X)}$ is again in seconds.
The simple answer is that the units are on the same scale as the mean. Example: I estimate the mean for secondary student to be 160cm with a standard deviation (SD) of 20cm. It is intuitively easier to get a sense of the variation with the SD than the variance of 400cm^2.
In more simple terms standard deviation is designed to give us a positive number that says something about the spread of our data about it's mean.
If we were to just add up the distances of all the points from the mean, then points in the positive and negative directions would combine in a way that would tend to gravitate back toward the mean and we would lose information about the spread. This is why we measure variance first, so that all of the distances are preserved as positive quantities via squaring and they don't cancel each other out. In the end we want a positive value that represents the units we started with - this has already been commented on above - so we take the positive square root.
It is a historical stupidity which we continue due to intellectual laziness. They chose to square the differences from the mean in order to get rid of the minus sign. Then they took the square root so as to bring it to a scale similar to the mean.
Someone should generate new statistics, computing variance and SD using modulus or absolute values of deviance from the mean. This would get rid of this whole squaring and then taking the square root business.
