Is there a simple way of detecting outliers?

Question

I am wondering if there is a simple way of detecting outliers.

For one of my projects, which was basically a correlation between the number of times respondents participate in physical activity in a week and the number of times they eat outside the home (fast food) in a week, I drew a scatterplot and literally removed the data points that were extreme. (The scatterplot showed a negative correlation.)

This was based on value judgement (based on the scatterplot where these data points were clearly extreme). I did not do any statistical tests.

I am just wondering if this is a sound way of dealing with outliers.

I have data from 350 people so loss of (say) 20 data points is not a worry to me.

Answer 1

There is no simple sound way to remove outliers. Outliers can be of two kinds:

1) Data entry errors. These are often the easiest to spot and always the easiest to deal with. If you can find the right data, correct it; if not, delete it.

2) Legitimate data that is unusual. This is much trickier. For bivariate data like yours, the outlier could be univariate or bivariate.

a) Univariate. First, "unusual" depends on the distribution and the sample size. You give us the sample size of 350, but what is the distribution? It clearly isn't normal, since it's a relatively small integer. What is unusual under a Poisson would not be under a negative binomial. I'd kind of suspect a zero-inflated negative binomial relationship.

But even when you have the distribution, the (possible) outliers will affect the parameters. You can look at "leave one out" distributions, where you check if data point q would be an outlier if the data had all points but q. Even then, though, what if there are multiple outliers?

b) Bivariate. This is where neither variable's value is unusual in itself, but together they are odd. There is a possibly apocryphal report that the census once said there were 20,000 12 year old widows in the USA. 12 year olds aren't unusual, widows aren't either, but 12 year old widows are.

Given all this, it might be simpler to report a robust measure of relationship.

Answer 2

I have done a lot of research on outliers, particularly when I worked on energy data validation at Oak Ridge from 1978 to 1980. There are formal tests for univariate outliers for normal data (e.g. Grubbs' test and Dixon's ratio test). There are tests for multivariate outliers and time series. The book by Barnett and Lewis on "Outliers in Statistical Data" is the bible on outliers and covers just about everything.

When I was at Oak Ridge working on data validation we had large multivariate data sets. For univariate outliers, there is a direction for extremes (highly above the mean and highly below the mean). But for multivariate outliers there are many directions to look for outliers. Our philosophy was to consider what the intended use of the data is. If you are trying to estimate certain parameters such as a bivariate correlation or a regression coefficient then you want to look in the direction that provides the greatest effect on the parameter of interest. At that time I had read Mallows' unpublished paper on influence functions. The use of influence functions to detect outliers is covered in Gnanadesikan's multivariate analysis book. Of course, you can find it in Barnett and Lewis also.

The influence function for a parameter is defined at points in the multivariate space of the observations and essentially measures the difference between the parameter estimate when the data point is included compared with when it is left out. You can do such estimates with each sample point but usually, you can derive a nice functional form for the influence function that gives insight and faster computation.

For example in my paper in the American Journal of Mathematical and Management Science in 1982 "The Influence Function and Its Application to Data Validation" I show the analytic formula for the influence function for bivariate correlation and that the contours of constant influence are hyperbolae. So the contours show the direction in the plane where the influence function increases the fastest.

In my paper, I show how we applied the influence function for bivariate correlation with the FPC Form 4 data on generation and consumption of energy. There is a clear high positive correlation between the two and we found a few outliers that were highly influential on the estimate of correlation. Further investigation showed that at least one of the points was in error and we were able to correct it.

But an important point that I always mention when discussing outliers is that automatic rejection is wrong. The outlier is not always an error and sometimes it provides important information about the data. Valid data should not be removed just because it doesn't conform with our theory of reality. Whether or not it is difficult to do, the reason why the outlier occurred should always be investigated.

I should mention that this is not the first time multivariate outliers have been discussed on this site. A search for outliers would probably lead to several questions where multivariate outliers have been discussed. I know that I have referenced my paper and these books before and given links to them.

Also when outlier rejection is discussed many of us on this site have recommended against it especially if it is done based solely on a statistical test. Peter Huber often mentions robust estimation as an alternative to outlier rejection. The idea is that robust procedures will downweight the outliers reducing their effect on estimation without the heavy-handed step of rejecting them and using a non-robust estimator.

The influence function was actually originally developed by Frank Hampel in his PhD dissertation in the early 1970s (1974 I think). His idea was actually to use influence functions to identify estimators that were not robust against outliers and to help develop robust estimators.

Here is a link to a previous discussion on this topic where I mentioned some work of mine on detecting outliers in time series using influence functions.

Answer 3

Another simple approach to dealing with outliers is to use non-parametric statistics. Probably with your sample size a Spearman's rho would work well as an index of the correlation. (Note, though, that non-parametric, rank-order statistics do not help you much with non-linear relationships.)

If you want to use a Pearson's r (a parametric statistic), and if you are not able to compute Cook's distance, you might use a standard rule of thumb that any data point that is more than 2.67 standard deviations (s.d.) from the mean, or 4.67 s.d. from the mean is an outlier or extreme, respectively. These are typical cutoff values for outliers and extreme data points that are used in one standard statistical analysis program (SPSS).

Just because a data point is an outlier does not mean it is bad data to be discarded. You might compute your correlation with and without extreme points and go from there.

Answer 4

You might want to try Cook's Distance. See the wikipedia article for suggested cutoffs. Also, if you are heading toward some regression model, then you may wish to try robust regression.

Answer 5

Firstly, don't remove atypical values unless you are sure that are out of the study! They may contain some important information (variability). You should drop them if it's obvious that the outlier is due to incorrectly entered or measured data. If you don't know the sampling method used to obtain your data, then you should identify atypical values and their effects as follows:

1. Degree of abnormality: it is expected a 5% of the observations with standarized residuals ($e_i*$) > 2. If you have some more higher residuals you can suspect outliers.

2. Degree of distance to the center gravity in the space of x's: $h_{ii}$ (leverage). When some $h_{ii}$ is very high you have an observation that may distort your model because is out of the range of your study.

3. Degree of influence on the fitted model: Influential points are those which have enough weight to change your model. Then, cofficients of fitted model using all n observations are very different from coefficients of the fitted model using all points but not this observation $i$-th.
   Cook's distance or Cook's D is a commonly used estimate of the influence of a data point. : $DC_i=ei*^2·h_{ii}/[(1-h_{ii})·p]$

Possible solutions:

• Transforming variables and / or adding new variables to the model.
• For influential observations which are nothing but outliers, if not many, you can remove those individuals.

Answer 6

Distinguish identifying outliers (which many others have answered about) from removing them.

The outliers may be the most interesting finding in your study. Some people may eat fast food often and be much more physically active than most others. Why? Launching an initiative to figure that out may be the most important outcome of your study.

Outliers are unusual. That could be because of an error in data collection or recording, in which case removing from analysis might make sense. Or it could be due to some other reason that is worth tracking down! Don't remove the most interesting data in your study just because the values are unusual!