What books have told me that 'standard deviation' has a special relationship with normal distribution i.e. 68–95–99.7% rule. What they didn't tell me is the answer to "can I use standard deviation to calculate spread of any distribution?"

In that case I should be taking standard deviation of binomial or poisson etc. distribution or not?

  • $\begingroup$ What problem would you "use" the standard deviation to solve? What are you trying to do, and how does a standard deviation fit into it? $\endgroup$
    – Sycorax
    Dec 7 '20 at 0:48
  • $\begingroup$ To find the spread, variability etc. like you would do with normal distribution because 'mean' is not enough e.g. to compare two sets. $\endgroup$
    – user963241
    Dec 7 '20 at 1:52

Sort of.

It is possible to have a distribution for which the variance is undefined. Then it doesn’t make sense to take the square root to get the standard deviation.

When the variance is defined, then it makes perfect sense to calculate it. The normal distribution has that nice 68/95/99.7 property, which other distributions do not, so do not get misled into thinking that 68% of the density is within 1 standard deviation in general.

The Chebyshev inequality is valid whenever the variance is defined, however.

$$ P(\vert X-\mu\vert >k\sigma)\le \dfrac{1}{k^2} $$

$X$ is the random variable.

$\mu$ is the mean.

$\sigma$ is the standard deviation.

$k$ is a positive number.

So when variance is defined, there is some notion of higher standard deviation corresponds to being more spread out from the mean.

  • $\begingroup$ Do I need to check with this 'Chebyshev inequality' before proceeding to calculate the standard deviation? Maybe if I can do it directly in Python would be great. $\endgroup$
    – user963241
    Dec 7 '20 at 0:18
  • 1
    $\begingroup$ What would you check? $\endgroup$
    – Dave
    Dec 7 '20 at 0:38
  • $\begingroup$ In other words, do I need to calculate this 'Chebyshev inequality' before calculating the standard deviation? $\endgroup$
    – user963241
    Dec 7 '20 at 1:50
  • $\begingroup$ The Chebyshev inequality is a more general/less strict version of the 68/95/99.7 rule that you mention in your question: it tells you the probability that an observation is more than $k$ standard deviations from the mean. $\endgroup$
    – Sycorax
    Dec 7 '20 at 1:58
  • 1
    $\begingroup$ There is nothing to check and even if there were you would still need to calculate the SD to verify that the inequality is satisfied. $\endgroup$
    – Nick Cox
    Dec 7 '20 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.