28
$\begingroup$

Following my question here, I am wondering if there are strong views for or against the use of standard deviation to detect outliers (e.g. any datapoint that is more than 2 standard deviation is an outlier).

I know this is dependent on the context of the study, for instance a data point, 48kg, will certainly be an outlier in a study of babies' weight but not in a study of adults' weight.

Outliers are the result of a number of factors such as data entry mistakes. In my case, these processes are robust.

I guess the question I am asking is: Is using standard deviation a sound method for detecting outliers?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ You say, "In my case these processes are robust". Meaning what? That you're sure you don't have data entry mistakes? $\endgroup$
    – Wayne
    Commented Sep 26, 2012 at 12:01
  • $\begingroup$ There are so many good answers here that I am unsure which answer to accept! Any guidance on this would be helpful $\endgroup$
    – Amarald
    Commented Sep 27, 2012 at 0:02
  • $\begingroup$ In general, select the one that you feel answers your question most directly and clearly, and if it's too hard to tell, I'd go with the one with the highest votes. Even it's a bit painful to decide which one, it's important to reward someone who took the time to answer. $\endgroup$
    – Wayne
    Commented Sep 27, 2012 at 12:02
  • 1
    $\begingroup$ P.S. Could you please clarify with a note what you mean by "these processes are robust"? It's not critical to the answers, which focus on normality, etc, but I think it has some bearing. $\endgroup$
    – Wayne
    Commented Sep 27, 2012 at 12:03
  • 4
    $\begingroup$ Outliers are not model-free. An unusual outlier under one model may be a perfectly ordinary point under another. The first question should be "why are you trying to detect outliers?" (rather than do something else, like use methods robust to them), and the second would be "what makes an observation an outlier in your particular application?" $\endgroup$
    – Glen_b
    Commented Jun 20, 2015 at 7:28

4 Answers 4

Reset to default
29
$\begingroup$

Some outliers are clearly impossible. You mention 48 kg for baby weight. This is clearly an error. That's not a statistical issue, it's a substantive one. There are no 48 kg human babies. Any statistical method will identify such a point.

Personally, rather than rely on any test (even appropriate ones, as recommended by @Michael) I would graph the data. Showing that a certain data value (or values) are unlikely under some hypothesized distribution does not mean the value is wrong and therefore values shouldn't be automatically deleted just because they are extreme.

In addition, the rule you propose (2 SD from the mean) is an old one that was used in the days before computers made things easy. If N is 100,000, then you certainly expect quite a few values more than 2 SD from the mean, even if there is a perfect normal distribution.

But what if the distribution is wrong? Suppose, in the population, the variable in question is not normally distributed but has heavier tails than that?

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ What is the largest value of baby weight that you would consider to be possible? $\endgroup$
    – mark999
    Commented Sep 26, 2012 at 11:52
  • 2
    $\begingroup$ I don't know. But one could look up the record. According to answers.com (from a quick google) it was 23.12 pounds, born to two parents with gigantism. If I was doing the research, I'd check further. $\endgroup$
    – Peter Flom
    Commented Sep 26, 2012 at 17:13
  • 1
    $\begingroup$ What if one cannot visually inspect the data (i.e. it might be part of an automatic process?) $\endgroup$
    – user90772
    Commented Mar 10, 2017 at 14:41
  • $\begingroup$ Add graphs to the automation, somehow. $\endgroup$
    – Peter Flom
    Commented Mar 10, 2017 at 14:45
24
$\begingroup$

Yes. It is a bad way to "detect" oultiers. For normally distributed data, such a method would call 5% of the perfectly good (yet slightly extreme) observations "outliers". Also when you have a sample of size n and you look for extremely high or low observations to call them outliers, you are really looking at the extreme order statistics. The maximum and minimum of a normally distributed sample is not normally distributed. So the test should be based on the distribution of the extremes. That is what Grubbs' test and Dixon's ratio test do as I have mention several times before. Even when you use an appropriate test for outliers an observation should not be rejected just because it is unusually extreme. You should investigate why the extreme observation occurred first.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Just as "bad" as rejecting H0 based on low p-value. $\endgroup$
    – Leo
    Commented Sep 26, 2012 at 16:46
19
$\begingroup$

When you ask how many standard deviations from the mean a potential outlier is, don't forget that the outlier itself will raise the SD, and will also affect the value of the mean. If you have N values, the ratio of the distance from the mean divided by the SD can never exceed (N-1)/sqrt(N). This matters the most, of course, with tiny samples. For example, if N=3, no outlier can possibly be more than 1.155*SD from the mean, so it is impossible for any value to ever be more than 2 SDs from the mean. (This assumes, of course, that you are computing the sample SD from the data at hand, and don't have a theoretical reason to know the population SD).

The critical values for Grubbs test were computed to take this into account, and so depend on sample size.

Improve this answer
$\endgroup$
12
$\begingroup$

I think context is everything. For the example given, yes clearly a 48 kg baby is erroneous, and the use of 2 standard deviations would catch this case. However, there is no reason to think that the use of 2 standard deviations (or any other multiple of SD) is appropriate for other data. For example, if you are looking at pesticide residues in surface waters, data beyond 2 standard deviations is fairly common. These particularly high values are not “outliers”, even if they reside far from the mean, as they are due to rain events, recent pesticide applications, etc. Of course, you can create other “rules of thumb” (why not 1.5 × SD, or 3.1415927 × SD?), but frankly such rules are hard to defend, and their success or failure will change depending on the data you are examining. I think using judgment and logic, despite the subjectivity, is a better method for getting rid of outliers, rather than using an arbitrary rule. In this case, you didn't need a 2 × SD to detect the 48 kg outlier - you were able to reason it out. Isn't that a superior method? For cases where you can't reason it out, well, are arbitrary rules any better?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.