3
$\begingroup$

I've been asked to analyse the distribution of a set of data, essentially a single column of random samples of a physical parameter which cannot be negative. The standard deviation that I usually calculate assumes that data are normally distributed, which in this case cannot be true.

Since I'm used to being asked to provide the standard deviation and mean of a dataset as a measure of the errors, I'd like to know whether there is an easily calculated parameter for a more appropriate distribution that would fulfil the same purpose?

I apologise if this question is ill-posed; I am not a statistician by training.

With many thanks,

Loruschorus

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Because the standard deviation, as universally defined, makes no assumptions about normality--it's merely a sample statistic-- how does the SD that you "usually calculate" differ from it? $\endgroup$
    – whuber ♦
    Commented Dec 19, 2011 at 14:46
  • 2
    $\begingroup$ This kind of comment, that standard deviation is defined only for normal distributions, or makes sense only for normal distributions, or the formula for standard deviation assumes that the data set is normally distributed, shows up so frequently here that it makes me wonder whether there is a commonly used textbook for a STAT 101 type course (or a popular web site that people visit frequently) which makes such claims or, more likely, makes statements that lead readers to make such unwarranted inferences. $\endgroup$
    – Dilip Sarwate
    Commented Dec 19, 2011 at 16:12
  • 4
    $\begingroup$ @Dilip There are estimates of standard deviation, often called "pseudo SDs," that assume approximate normality. One is the IQR divided by 1.35. Another is half the difference between the 84th and 16th sample percentiles. A deeper question lurking here is whether an estimated SD is an appropriate way to measure variation for a strikingly non-normal distribution. (Another issue is the implicit, but incorrect, assumption that positive values cannot be adequately modeled by normal distributions.) $\endgroup$
    – whuber ♦
    Commented Dec 19, 2011 at 16:34
  • $\begingroup$ if, by positive the OP means asymetric and by Gaussian he means symmetric with square integrable tails then the OP has a point... $\endgroup$
    – user603
    Commented Mar 10, 2014 at 22:37
  • 1
    $\begingroup$ Does this answer your question? Can I use 'mean ± SD' for non-negative data when SD is higher than mean? $\endgroup$
    – mkt
    Commented May 9, 2023 at 10:13

2 Answers 2

Reset to default
5
$\begingroup$

Standard deviation is a measure that can be calculated on any set of data regardless of its actual distribution. It is simply a measure of value dispersion in relation to the data set's mean.

Any normality assumption to which you are referring usually is only a concern when doing statistical inference. For instance, if you need to test whether a sample standard deviation is 'large' or if two sample standard deviations are the same, then the underlying distribution of the data is important.

So, if all you need for your analysis is to do is compute the standard deviation, then the standard deviation formula you have been using is sufficient.

Improve this answer
$\endgroup$
3
$\begingroup$

If your data is in some odd distribution and you think mean + standard deviation is not doing a good job of describing your data, you can use some other measures like mode, median, max/min, interquartile range, etc.

I guess you need to tell us more about your data and how standard deviation is inappropriate before others can help with suggestions.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your comments (and to the other posters as well). I'm also sorry to those whom I have irritated by my relative ignorance. To clarify: when I referred to the standard method of calculating the standard deviation, I simply meant by calculating the variance of the data and taking the square root. My data are distributed more like a poisson or wald distribution, and I just wanted to check that I wouldn't be giving a misleading or inappropriate measure of the spread in my data by quoting the mean and standard deviation. $\endgroup$
    – Jeff Winterbourne
    Commented Dec 20, 2011 at 7:59
  • $\begingroup$ (hit the character limit - sorry for the double post) I will take your advice and quote the interquartile range, as I cannot see how that can be misinterpreted. Thanks again for your help $\endgroup$
    – Jeff Winterbourne
    Commented Dec 20, 2011 at 8:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.