How to deal with outliers?

Question

Background:
There are often some values among sampling set that appear not closely compatible with the rest. They would be called as extreme values or simply outliers. Dealing with outliers has been always a matter of challenge. There are some approaches to solve the problem of the existence of outliers:

• moving them to a separated set
• replacing them with nearest values from non-outlier set
• ...

Question:
What is the most recommended method(s) to deal with outliers? (with details and an example)

Answer 1

I've recommended two methods in the past. They depend on the nature of the data in a general sense.

If the outliers are part of a well known distribution of data with a well known problem with outliers then, if others haven't done it already, analyze the distribution with and without outliers, using a variety of ways of handling them, and see what happens. You're going to be dealing with this data a lot. You might as well understand an outlier problem. For example, Ratcliff has a nice little paper reaction times that you might look at as an example. If there are papers like that for your example then read them.

If the outliers are from a data set that is relatively unique then analyze them for your specific situation. Analyze both with and without them, and perhaps with a replacement alternative, if you have a reason for one, and report your results of this assessment.

So, in short, analyze and document. That's the best thing to do.