Outlier removal prior to mixed-effect modelling

Question

I'm analysing reaction time data from a grammaticality judgement task (collected in a masked-priming experiment). The stimulus were noun-noun compounds, including 3 types of compounds (depending on semantic relation). Each compound was tested 4 times, in a 2x2 design (prime = N1 or N2; order = grammatical or ungrammatical). Participants included native and non-native speakers.

I have removed items with physically impossibly short RTs and latencies excluding 5 seconds, and am now concerned that more subtle outliers should be removed prior to analysis. Following Baayen & Milin (2010) [pdf], I have transformed reaction times as 1/RT. They suggest that "if the precondition of normality is well met, [...] outlier removal before model fitting is not necessary". My data is not normally distributed when considered by subject (the Shapiro test indicates only 1 in 21 subjects yields p > 0.05), but I guess this is to be expected given the design?

Should I screen the data for outliers prior to model fitting?

@article{
   Author = {Baayen, R. Harald and Milin, Petar},
   Title = {Analysing Reaction Times},
   Journal = {International Journal of Psychological Research},
   Volume = {3},
   Number = {2},
   Pages = {12--28},
      Year = {2010} }

Answer 1

Having taken a look at paper cited, it's not quite as bad as I thought -- they basically suggest normality testing as a way to identify extreme outliers that might screw up the analysis, and they say that mixed modeling allows less "aggressive" outlier identification/removal. (They use the terms "minimal trimming" and "mild" vs "aggressive" a priori data screening/outlier removal, which at a quick glance I don't see defined precisely in the paper: perhaps they're defined in the references??)

At least pre-screening according to strictly specified rules, based only on the response variable without taking the predictors into account, does not lead to the danger of data snooping (as opposed to, say, running an initial analysis without screening and then going back and screening only if you find the results not to your liking).

I still don't like it though, and would probably say that only results that are qualitatively robust to the presence or absence of outliers should be taken completely seriously.) However, my previous answer on r-help still holds:

• outlier detection etc. should be done on the conditional distributions, not marginal distributions -- given a strong effect of a qualitative predictor, the marginal distribution will be multimodal = not normal at all (I don't see how Baayen and Milin get away with this);
• as you suggest, 1/21 values of $p<0.05$ is very close to the expectation in the null case.

I would personally prefer that you skip a priori test-based screening completely (removing physically impossible/very long values as suggested in the paper is perfectly sensible) and use model criticism instead (as suggested by the paper), but you do have to conform to the norms of your community where they are not completely outrageous. If possible, compare your results with screened vs. unscreened data and see that they are qualitatively similar.

Answer 2

If you are absolutely certain that nobody will ever critically review your analysis, consider it skeptically, or just need to be convinced of your results, then go ahead and remove the outliers (but if this is the case then there is probably no need to do the analysis at all).

If you remove outliers then you open yourself to accusations of cherry-picking only those data points that confirm your preconceptions.

If you are concerned about the normality assumptions (and note that it is the residuals, not the response that need to be normal) then you should consider a method that uses all the data, but does not depend on normality rather than deleting points (e.g. data tranformations, permutation tests, bootstrap, or other non-parametric approaches).

If you have points that are true outliers without obvious explanation, then examining why they are different may lead to more interesting findings than the original analysis.

Note that the common tests of normality generally have poor power to find important differences when they matter and high power to find minor differences when they don't matter. Decisions about using normal based inference or not should come from your knowledge (and that of other researchers) of the science that produces the data. While the canned normality tests were developed by people who were probably smarter than me and possibly smarter than you, it cannot be expected of them that they knew (at the time they developed the test) more about your data and your questions than you do.