45
$\begingroup$

In a normal distribution, the 68-95-99.7 rule imparts standard deviation a lot of meaning, but what would standard deviation mean in a non-normal distribution (multimodal or skewed)? Would all data values still fall within 3 standard deviations? Do we have rules like the 68-95-99.7 one for non-normal distributions?

$\endgroup$
  • 17
    $\begingroup$ Have a look at Chebyshev's inequality. $\endgroup$ – COOLSerdash Jul 20 '14 at 8:08
  • $\begingroup$ @COOLSerdash great. This perfectly answers my question. $\endgroup$ – Zuhaib Ali Jul 20 '14 at 8:38
  • 4
    $\begingroup$ @COOLSerdash's point is on-target here, but be aware that the standard statement of Chebyshev's inequality pertains to the true SD known a-priori, not an SD estimated from your sample. It may help to read this excellent CV thread: Does a sample version of the one-sided Chebeshev inequality exist? $\endgroup$ – gung - Reinstate Monica Jul 20 '14 at 16:15
  • $\begingroup$ Also, you should probably not settle for Chebyshev right away--you can probably do a lot better, skewed or not. $\endgroup$ – Steve S Jul 20 '14 at 16:34
  • 1
    $\begingroup$ @gung so does the 68-95-99.7 rule! $\endgroup$ – Glen_b -Reinstate Monica Jul 20 '14 at 19:52
12
$\begingroup$

The standard deviation is one particular measure of the variation. There are several others, Mean Absolute Deviation is fairly popular. The standard deviation is by no means special. What makes it appear special is that the Gaussian distribution is special.

As Pointed out in comments Chebyshev's inequality is useful for getting a feeling. However there are a more.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Re "by no means:" arguably, what makes the SD so special is its close relation with the variance which is (asymptotically, of course) the unique measure of spread that makes the Central Limit Theorem work. $\endgroup$ – whuber Jan 14 at 18:37
  • $\begingroup$ But only because the Central Limit Theorem leads to a Gaussian. But sure, maybe my language was stronger than it needed to be. $\endgroup$ – Keith Jan 14 at 19:14
  • $\begingroup$ I think that confuses cause and effect, Keith. At stats.stackexchange.com/a/3904/919 I have attempted to explain the primacy of the SD without invoking the conclusion of the theorem. The point is that the mean and SD play a role in achieving a sequence of random variables that converges in distribution, regardless of what that limit might be: its Normality is a separate issue. $\endgroup$ – whuber Jan 14 at 19:17
  • 1
    $\begingroup$ Sure. E T Jaynes covers why the normal function comes up in this specific case in "Probability Theory: The Logic of Science" $\endgroup$ – Keith Jan 14 at 23:52
12
$\begingroup$

It's the square root of the second central moment, the variance. The moments are related to characteristic functions(CF), which are called characteristic for a reason that they define the probability distribution. So, if you know all moments, you know CF, hence you know the entire probability distribution.

Normal distribution's characteristic function is defined by just two moments: mean and the variance (or standard deviation). Therefore, for normal distribution the standard deviation is especially important, it's 50% of its definition in a way.

For other distributions the standard deviation is in some ways less important because they have other moments. However, for many distributions used in practice the first few moments are the largest, so they are the most important ones to know.

Now, intuitively, the mean tell you where the center of your distribution is, while the standard deviation tell you how close to this center your data is.

Since the standard deviation is in the units of the variable it's also used to scale other moments to obtain measures such as kurtosis. Kurtosis is a dimensionless metric which tells you how fat are the tails of your distribution compared to normal

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ "Now, intuitively, the mean tell you where the center of your distribution is, while the standard deviation tell you how close to this center your data is." - Wouldn't this only apply if the distribution is Normal? Otherwise, more often than not, the median is a better measure of central tendency. $\endgroup$ – Dan Temkin Feb 6 '18 at 21:16
  • 1
    $\begingroup$ @DanTemkin When using the median, the standard deviation loses its value to a degree. It's calculated off of the mean. With median then it makes a sense to talk about quantiles, which could be a way to go with skewed distributions. OP didn't focus on skewed distributions though. So, for any symmetrical distribution you have mean=median, it doesn't have top be normal. Thus it makes a sense to talk about mean when standard deviation is discussed. $\endgroup$ – Aksakal Feb 6 '18 at 21:19
  • $\begingroup$ @DanTemkin No, it is not. What you just said effectively implies "Only use the median, unless the mean equals the median." The median and mean are simply different ways of defining central tendency, each with their own value. $\endgroup$ – Alexis Jan 14 at 18:02
5
$\begingroup$

The sample standard deviation is a measure of the deviance of the observed values from the mean, in the same units used to measure the data. Normal distribution, or not.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.