10
$\begingroup$

I'm wondering if the standard deviation was always built on the assumption of a normal distribution. In other words, if the sample is not normally distributed, then should using the standard deviation be considered as a mistake?

$\endgroup$
3
  • 4
    $\begingroup$ A uniform distribution has a standard deviation, how could this be a "mistake"? $\endgroup$ – Carlo Beenakker Mar 23 '17 at 10:34
  • $\begingroup$ Why would it be? The standard deviation is just a measure of how much deviation there is in a set of numbers, you can find the standard deviation of "1, 2, 5, 14". Standard deviation wasn't "built" by anyone, it's just a statistical property. $\endgroup$ – user279040 Apr 11 at 22:20
  • $\begingroup$ Not necessarily but using the standard deviation to describe skewed distributions can make less sense. Maybe that's where your question is coming from. $\endgroup$ – Michael Webb Apr 11 at 22:25
20
$\begingroup$

No. The use of standard deviation does not assume normality.

The variance of a random variable is defined as $\operatorname{Var}(X) = \operatorname{E}[(X - \operatorname{E}[X])^2]$. As long as the variance exists, the standard deviation also exists. The standard deviation is the square root of the variance.

You can use the variance $\operatorname{Var}(X)$ or standard deviation any time that the two exist. The variance comes up in countless situations.

There are special theorems, lemmas etc... though for the special case where $X$ follows the normal distribution.

A common use of standard deviation that does depend on normality:

If $X$ follows the normal distribution, then there's approximately a 95% probability that $X$ falls within two standard deviations of the mean.

That statement is true if $X$ follows the normal distribution (and several others) but it isn't true in general.

A common use of the variance that does not depend on normality:

Let $X$ be a random variable with mean $\operatorname{E}[X] = \mu$ and variance $\operatorname{Var}(X) = \sigma^2$. Define $X_i$ for $i=1, \ldots, n$ as independent random variables, each following the identical distribution as $X$.

Define the sample mean based upon $n$ observations as: $$\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$$

By the Central Limit Theorem, $\bar{X}_n$ converges towards a normally distributed random variable with mean $\mu$ and variance $\frac{\sigma^2}{n}$. (More precisely $\sqrt{n}\left( \bar{X}_n - \mu \right)$ converges in distribution to $\mathcal{N}(0,\sigma^2)$ as $n \rightarrow \infty$.)

The practical implication is that the sample mean $\bar{X}_n$ for large $n$ can be treated as normally distributed random variable whose variance $\frac{\sigma^2}{n}$ is a function of the variance of $X$. (Recall $\operatorname{Var}(X)=\sigma^2$.) And this result does not require that $X$ be normal. (It does require a lower $n$ to work well if $X$ is closer in some sense to the normal distribution though.)

The Central Limit Theorem is a ubiquitous tool that uses the variance of $X$ and does not need $X$ to follow the normal distribution.

$\endgroup$
3
  • 4
    $\begingroup$ Chebyshev's Inequality is not specific to the variance: an equally useful version exists for every absolute moment with power greater than $1$. I would therefore suggest looking elsewhere for reasons why the SD is important and (almost) universal, such as the unique role played by variance in the Central Limit Theorem. $\endgroup$ – whuber Mar 23 '17 at 14:02
  • $\begingroup$ @whuber Yeah, I had started writing a CLT example (and now I've added it). The CLT is an extremely practical reason to care about the variance. $\endgroup$ – Matthew Gunn Mar 23 '17 at 14:47
  • 1
    $\begingroup$ +1. But note that while variance (together with the mean) gives a complete description in the normal case, for nonnormal distribution this might no longer be the case, and other d3scriptors of the data might be much better $\endgroup$ – kjetil b halvorsen Mar 23 '17 at 23:17
3
$\begingroup$

In the standard IID setting, under suitable regularity conditions, $S^2$ (as well as $\hat{\sigma}^2_{ML}$) is a strongly consistent estimator of $\mathrm{Var}[X_i]$. This follows directly from the Strong Law of Large Numbers. A normal model assumption is not needed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy