I'm wondering how useful the standard deviation is when applied to positively skewed data? The standard deviation implies that 68% of data will lie within one standard deviation of the mean, but surely this will not work for positively skewed data? This page suggests that for positively skewed data, the standard deviation is not useful and quartiles should be used instead.
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ 68% of the normal mean... $\endgroup$– Xi'anCommented Dec 26, 2014 at 9:37
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
I'm wondering how useful the standard deviation is when applied to positively skewed data?
It can be quite useful but it depends on what you're doing.
For example, I use the lognormal and gamma distributions regularly and sometimes use the standard deviation with those distributions -- it can be informative in some contexts.
The standard deviation implies that 68% of data will lie within one standard deviation of the mean,
No, that's true for normal populations. It's not something that holds more generally - even with symmetric distributions you can have almost nothing, or anything up to 100% within one standard deviation of the mean.
but surely this will not work for positively skewed data?
Again, it depends on the distribution - you might get more or less than 68% with positively skewed or negatively skewed distributions.
This page suggests that for positively skewed data, the standard deviation is not useful and quartiles should be used instead.
It depends on what you're trying to do. Distributions needn't be skew for the standard deviation not to be suitable, and the standard deviation may be quite okay when distributions are skew.
-
1$\begingroup$ Further examples: Binomial distributions can be highly skewed and Poisson distributions will always be skewed (bar the limit of a normal) but mean and SD or variance remain the measures to use with both, theoretically and practically. The median and IQR for such distributions can be of descriptive use occasionally but otherwise they can be awkward at best. $\endgroup$– Nick CoxCommented Sep 28, 2021 at 0:29
$\begingroup$
$\endgroup$
3
Expanding a little on @glen_b 's good answer: Usually the mean (as a measure of central tendency) goes with the SD (as a measure of spread).
When a distribution is highly skew, we often do not want the mean, but sometimes we do. Take income. This is usually reported as median family income, precisely because it is highly skew. But there are times when the mean income is what you need - e.g. if you are measuring the total amount of money going into a family, clan or other group. If (say) in a family the mother makes \$60,000, the father makes \$50,000 and the 3 kids each make \$0, it may be good to know the mean income rather than the median. With 5 people you don't really need measures of spread, but if you were doing this for a lot of families, it might be useful.
answered Dec 26, 2014 at 11:51
-
2$\begingroup$ the Latex dollars have struck again! (Nice answer by the way.) $\endgroup$ Commented Dec 26, 2014 at 12:15
-
$\begingroup$ Thanks @Silverfish for both the note about the dollars and the compliment. (I fixed the former). $\endgroup$ Commented Dec 26, 2014 at 12:42
-
2$\begingroup$ I think quoting "mean with sd", "median with IQR" and noting that for skewed data we often but don't always use the median is a good summary for a learner. Perhaps it'd be a good idea to say why we pair our central tendency and spread summaries in this way - based on moments vs based on quantiles. $\endgroup$ Commented Dec 26, 2014 at 12:53
$\begingroup$
$\endgroup$
2
Not really useful. One calculates the standard deviation of the mean to have an idea of how precise measurements are. But you are after a different question.
Better statistics for this problem are the interquartile range (with is associated representation the box plot), and the skewness. They explicitly address this question. Concretely, the skewness is the 3rd moment of the distribution. If it were perfectly symmetric, this moment is zero. Otherwise it is either positive of negative, depending on whether you have more data on the right side of the mean value or the left, resp. The bigger (resp. the lower) the value of the skewness score, the higher variation of the data in the positive (resp. negative) direction.
-
1$\begingroup$ Could you explain yourself a little more fully? "Better" statistics in what sense and for what purpose? $\endgroup$– whuber ♦Commented Dec 26, 2014 at 13:44
-
$\begingroup$ If a moment-based skewness measure is satisfactory -- which is built on ideas like that of mean and SD -- then it is hard to see that mean and SD are unsatisfactory. It would be nice to be able to give simple neat advice, but there are many skewed situations where mean and SD are natural and many others where using them is a bad idea. $\endgroup$– Nick CoxCommented Sep 28, 2021 at 0:24