There is no clear answer in the literature on how to deal with the fact that RT data do not conform to the traditional Gaussian models of error. Some folks will say that one need not worry about non-Gaussian error within cells of the experimental design because traditional analysis approaches typically collapse these observations to a mean, and we know that the sampling distribution of the mean will tend to conform to the Gaussian assumption of error. Such arguments miss the point, now well established in the RT literature, that experimental variables can affect not only the central tendency of the RT distribution but also the scale and shape of the distribution. Indeed, a number of reports show that the traditional "collapse to a mean" approach can miss phenomena when a variable decreases the central tendency but increases skew.
For those interested in characterizing the full nature of the RT distribution (or at least, features of the distribution beyond central tendency), there are three general approaches:
- Quantify the distribution via quantiles (typically
seq(.1,.9,.2)) and add quantile to the list of fixed effect variables in your analysis. This requires that you have some method for dealing with the continuous-yet-likely-nonlinear nature of quantile as a variable; I like generalized additive mixed effects modelling for this purpose.
- Choose an a priori distribution form (ex. Ex-Gaussian, Wald, Weibull, etc) and attempt to estimate the best fitting parameters of this distribution (and the effect of your experimental variables thereon) given the data. While some folks will obtain parameter estimates per cell of the experimental design then submit the estimates to ANOVA, but where the assumption of Gaussian error may not be appropriate for these parameter estimates, I'd say that hierachircal modelling (as in Rouder et al 2005) is a better approach.
- Choose an a priori process model (ex. diffusion, linear ballistic accumulator, etc) and fit it repeatedly to the data, comparing the quality of the resulting fits as a function of whether you let certain parameters of interest to vary as a function of the experimental design. Again, this is best done in a hierarchical manner.
Personally, I like #3 but suggest that #1 should always be done as well to guard against the possibility that the process model isn't appropriate.
Now, how to do any of the above and account for outliers (fast and slow) is another matter, though Trisha Van Zandt had a neat talk at the SCiP meeting last year where she showed an approach where fast and slow outliers were explicitly modelled with a priori distributions (her modelling also accounted for serial correlations in RTs nicely).
