40
$\begingroup$

In a normal distribution, the 68-95-99.7 rule imparts standard deviation a lot of meaning, but what would standard deviation mean in a non-normal distribution (multimodal or skewed)? Would all data values still fall within 3 standard deviations? Do we have rules like the 68-95-99.7 one for non-normal distributions?

$\endgroup$
  • 16
    $\begingroup$ Have a look at Chebyshev's inequality. $\endgroup$ – COOLSerdash Jul 20 '14 at 8:08
  • $\begingroup$ @COOLSerdash great. This perfectly answers my question. $\endgroup$ – Zuhaib Ali Jul 20 '14 at 8:38
  • 3
    $\begingroup$ @COOLSerdash's point is on-target here, but be aware that the standard statement of Chebyshev's inequality pertains to the true SD known a-priori, not an SD estimated from your sample. It may help to read this excellent CV thread: Does a sample version of the one-sided Chebeshev inequality exist? $\endgroup$ – gung - Reinstate Monica Jul 20 '14 at 16:15
  • $\begingroup$ Also, you should probably not settle for Chebyshev right away--you can probably do a lot better, skewed or not. $\endgroup$ – Steve S Jul 20 '14 at 16:34
  • 1
    $\begingroup$ @gung so does the 68-95-99.7 rule! $\endgroup$ – Glen_b -Reinstate Monica Jul 20 '14 at 19:52
12
$\begingroup$

The standard deviation is one particular measure of the variation. There are several others, Mean Absolute Deviation is fairly popular. The standard deviation is by no means special. What makes it appear special is that the Gaussian distribution is special.

As Pointed out in comments Chebyshev's inequality is useful for getting a feeling. However there are a more.

$\endgroup$
9
$\begingroup$

It's the square root of the second central moment, the variance. The moments are related to characteristic functions(CF), which are called characteristic for a reason that they define the probability distribution. So, if you know all moments, you know CF, hence you know the entire probability distribution.

Normal distribution's characteristic function is defined by just two moments: mean and the variance (or standard deviation). Therefore, for normal distribution the standard deviation is especially important, it's 50% of its definition in a way.

For other distributions the standard deviation is in some ways less important because they have other moments. However, for many distributions used in practice the first few moments are the largest, so they are the most important ones to know.

Now, intuitively, the mean tell you where the center of your distribution is, while the standard deviation tell you how close to this center your data is.

Since the standard deviation is in the units of the variable it's also used to scale other moments to obtain measures such as kurtosis. Kurtosis is a dimensionless metric which tells you how fat are the tails of your distribution compared to normal

$\endgroup$
  • 1
    $\begingroup$ "Now, intuitively, the mean tell you where the center of your distribution is, while the standard deviation tell you how close to this center your data is." - Wouldn't this only apply if the distribution is Normal? Otherwise, more often than not, the median is a better measure of central tendency. $\endgroup$ – Dan Temkin Feb 6 '18 at 21:16
  • $\begingroup$ @DanTemkin When using the median, the standard deviation loses its value to a degree. It's calculated off of the mean. With median then it makes a sense to talk about quantiles, which could be a way to go with skewed distributions. OP didn't focus on skewed distributions though. So, for any symmetrical distribution you have mean=median, it doesn't have top be normal. Thus it makes a sense to talk about mean when standard deviation is discussed. $\endgroup$ – Aksakal Feb 6 '18 at 21:19
4
$\begingroup$

The sample standard deviation is a measure of the deviance of the observed values from the mean, in the same units to measure the data. Normal distribution, or not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.