I'm analysing reaction time data from a grammaticality judgement task (collected in a masked-priming experiment). The stimulus were noun-noun compounds, including 3 types of compounds (depending on semantic relation). Each compound was tested 4 times, in a 2x2 design (prime = N1 or N2; order = grammatical or ungrammatical). Participants included native and non-native speakers.
I have removed items with physically impossibly short RTs and latencies excluding 5 seconds, and am now concerned that more subtle outliers should be removed prior to analysis. Following Baayen & Milin (2010) [pdf], I have transformed reaction times as 1/RT. They suggest that "if the precondition of normality is well met, [...] outlier removal before model fitting is not necessary". My data is not normally distributed when considered by subject (the Shapiro test indicates only 1 in 21 subjects yields p > 0.05), but I guess this is to be expected given the design?
Should I screen the data for outliers prior to model fitting?
@article{
Author = {Baayen, R. Harald and Milin, Petar},
Title = {Analysing Reaction Times},
Journal = {International Journal of Psychological Research},
Volume = {3},
Number = {2},
Pages = {12--28},
Year = {2010} }