# Is there a simple way of detecting outliers?

I am wondering if there is a simple way of detecting outliers.

For one of my projects, which was basically a correlation between the number of times respondents participate in physical activity in a week and the number of times they eat outside the home (fast food) in a week, I drew a scatterplot and literally removed the data points that were extreme. (The scatterplot showed a negative correlation.)

This was based on value judgement (based on the scatterplot where these data points were clearly extreme). I did not do any statistical tests.

I am just wondering if this is a sound way of dealing with outliers.

I have data from 350 people so loss of (say) 20 data points is not a worry to me.

-
See also the answers to the similar question rigorous definition of an outlier –  Jonas Sep 24 '12 at 3:51
Also very closely related is stats.stackexchange.com/questions/175. Many potential outlier detection methods are described in answers at stats.stackexchange.com/questions/213. But more to the point would be some context: what are you doing with this scatterplot? What conclusions are you trying to draw from it? Some conclusions will depend little on what you do with outliers, whereas others might depend on them critically. This indicates that the methods you use to identify and treat outliers must depend on the intended analysis. –  whuber Sep 24 '12 at 14:38
In economics datasets the standard practice is just to say "We windsorize the dataset at 2.5% and 97.5%", or alternatively 1% and 99%. Then they just delete the observations that are out of that quantile range. –  user14281 Sep 27 '12 at 8:43
@Harokitty Winsorising seems to mean clipping the values rather than discarding them. –  Peter Wood Oct 18 '12 at 14:40

There is no simple sound way to remove outliers. Outliers can be of two kinds:

1) Data entry errors. These are often the easiest to spot and always the easiest to deal with. If you can find the right data, correct it; if not, delete it.

2) Legitimate data that is unusual. This is much trickier. For bivariate data like yours, the outlier could be univariate or bivariate.

a) Univariate. First, "unusual" depends on the distribution and the sample size. You give us the sample size of 350, but what is the distribution? It clearly isn't normal, since it's a relatively small integer. What is unusual under a Poisson would not be under a negative binomial. I'd kind of suspect a zero-inflated negative binomial relationship.

But even when you have the distribution, the (possible) outliers will affect the parameters. You can look at "leave one out" distributions, where you check if data point q would be an outlier if the data had all points but q. Even then, though, what if there are multiple outliers?

b) Bivariate. This is where neither variable's value is unusual in itself, but together they are odd. There is a possibly apocryphal report that the census once said there were 20,000 12 year old widows in the USA. 12 year olds aren't unusual, widows aren't either, but 12 year old widows are.

Given all this, it might be simpler to report a robust measure of relationship.

-
+1, there's a lot of really good advice here. –  gung Sep 24 '12 at 1:19
Thanks. I am thinking perhaps a confidence ellipse would be a good indicator of outliers as it would show the percent of data that would lie within a certain confidence level (given bivariate normal distribution). –  Amarald Sep 24 '12 at 4:19
Your data cannot be bivariate normal, since it is composed of non-negative integers –  Peter Flom Sep 24 '12 at 10:01

I have done a lot of research on outliers, particularly when I worked on energy data validation at Oak Ridge from 1978 to 1980. There are formal tests for univariate outlier for normal data (e.g. Grubbs' test and Dixon's ratio test). There are tests for multivariate outliers and time series. The book by Barnett and Lewis' "Outliers in Statistical Data" that is the bible on outliers covers just about everything.

When I was at Oak Ridge working on data validation we had large multivariate data sets. For univariate outliers there is a direction for extremes (highly above the mean and highly below the mean). But for multivariate outliers there are many directions to look for oultiers. Our philosophy was to consider what the intended use of the data is. If you are trying to estimate certain parameters such as a bivariate correlation or a regression coefficient then you want to look in the direction that provides the greatest effect on the parameter of interest. At that time I had read Mallows unpublished paper on influence functions. The use of influence functions to detect outliers is covered in Gnanadesikan's multivariate analysis book. Of course you can find it in Barnett and Lewis also.

The influence function for a parameter is defined at points in the multivariate space of the observations and essentially measures the difference between the parameter estimate when the data point is included compared to when it is left out. You can do such estimates with each sample point but usually you can derive a nice functional form for the influence function that gives insight and faster computation.

For example in my paper in the American Journal of Mathematical and Management Science in 1982 "The Influence Function and Its Application to Data Validation" I show what the analytic formula for the influence function for bivariate correlation is and that the contours of constant influence are hyperbolae. So the contours show the direction in the plane where the influence function increases the fastest.

In my paper I show how we applied the influence function for bivariate correlation with the FPC Form 4 data on generation and consumption of energy. There is a clear high positive correlation between the two and we found a few outliers that were highly influential on the estimate of correlation. Further investigation showed that at least one of the points had an error in it and we were able to correct it.

But an important point that I always mention when discussing outliers is that automatic rejection is wrong. The outlier is not always an error and sometimes it provides important information about the data. Valid data should not be removed just because it doesnt conform with our theory of reality. Whether it is difficult to do or not the reason why the outlier occurred should always be investigated.

I should mention that this is not the first time multivariate outliers have been discussed in this site a search fir outliers woll brobably lead to several questions where multivariate outliers have been discussed . I know that I have referenced my paper and these books before and given links to them.

Also when outlier rejection is discuss many of us on this site have recommended against it especially if it is done based solely on a statistical test. Peter Huber often mentions robust estimation as an alternative to outlier rejection. The idea is that robust procedures will downweight the outliers reducing their effect on estimation without the heavy-handed step of rejecting them and using a non-robust estimator.

The influence function was actually originally developed by Frank Hampel in his PhD dissertation in the early 1970s (1974 I think). His idea was actually to use influence functions to identify estimators that were not robust qgainst outliers and to help develop robust estimators.

Here is a link to a previous discussion on this topic where I mentioned some work of mine on detecting outliers in time series using influence functions. A search of the site for outliers will reveal many related questions and some that were closed because they were duplicates. There should also be at least one where my paper in AJMMS is mentioned.

-

You might want to try Cook's Distance. See the wikipedia article for suggested cutoffs. Also, if you are heading toward some regression model, then you may wish to try robust regression.

-
This seems more like a comment than an answer; answers are usually lengthier and more detailed. E.g. if you included reasoning as to why Cook's distance is a good test for outliers and so one, this would be an answer. –  Peter Flom Sep 29 '12 at 19:36