what to do when outliers are identified?

Question

In one task that measures the times that participants take to respond to each item of a task some of the response times were considered outliers as they are more than 3SD above the mean of the participants' responses. I was advised to remove the outliers and sum the other results in order to obtain the total amount of time spent in the task. However, I do not want to just eliminate those values as I think that would not be a reliable method of representing the child's responses and would obviously change their results, so how does one do it when dealing with outliers? Is it ok just to take them? I ask this because I want to compare the sums of each child, thus I do not think that just eliminating the outliers is an ok prodecure.

I do not think it represents bad data as the participants tend to take longer to respond to items they consider more difficult, but they take more than 3SD above the mean to give a response, thus not being that representative of the overall time they take to respond. So I feel like it is a valuable information but I do not know if leaving the results untreated would be the most suitable method when the goal is to compare groups on response times.

Answer 1

Important distinction: are the outliers likely spurious or not?

It's important to keep in mind what's likely driving the outliers. Depending on what you're trying to do and what generates the outliers, you might want to reduce the effect of outliers on your analysis or you might not. A big question is, "are the outliers spurious?" Possible drivers of outliers are:

• Measurement error (you probably don't want these observations to drive your results):
  • In accounting, data is sometimes misrecorded. For example 10,200 might be incorrectly recorded as 102,000.
  • In your case, it sounds like slow response times might be due to kids simply not paying attention?
• Extreme outcomes (these observations may be critical to your results!):
  • In finance, bankruptcy of a company that issued highly rated bonds might be rare, an extreme outlier, but this isn't bad data!
  • Car crashes, fatalities, etc... might be extreme outliers, but these outliers are likely the absolutely most important observations!
  • In your case, I might imagine extreme slow response times might be due to kids with severe developmental disabilities?

Some standard techniques to reduce the influence of outliers:

1. Winsorizing the data:
   • With winsorizing at the 5 percent level, you would replace observations below the 5th percentile and above the 95h percentile with values of the 5th percentile and 95th percentile respectively. The basic idea here is that the outliers are extreme observations, but they're probably not as extreme as the numbers suggest.

2. Using a trimmed estimator:

   • This is a more extreme approach, possibly appropriate if the outliers are highly likely to be spurious. When making some calculation, the most extreme observations are simply excluded.

3. Use estimation techniques less sensitive to outliers. For example:

   • Quantile regression (eg. calculate median)
   • Minimize something like Huber loss instead of square error (ordinary linear regression minimizes the sum of squares and can be quite sensitive to outliers).

4. Use a transformation of your data (h/t Mudwarrior)

   • For example, the U.S. economy has numerous tiny companies and a number of huge companies. For many purposes, it's better to work with the logarithm of a company's market capitalization rather than the market capitalization itself.

Answer 2

Did you consider transforming your response times? Square root, fourth root and log transformations are often used to reduce the effects of outliers (see Underwood (1997) Experiments in Ecology).