How do I test differences in reaction times? I have data from an experiment in which participants had to respond to stimuli in two different conditions in a within-subject design. Obviously, I can fairly easily test for differences in accuracy between the two conditions. However, my initial idea was to check for differences in reaction times. When a subject incorrectly answers, I feel the associated reaction time is meaningless. What is the 'standard' way to do this?
 A: The traditional approach here has been to analyse accuracy and speed separately, so you would first compare the two conditions in terms of accuracy (paired-samples t test), and then discard the wrong answers, and compare the reaction times for correct answers from both conditions.
However, as you've said, there are obvious limitations to this approach: speed and accuracy interact in a myriad of ways, including but not limited to the well-known "speed-accuracy trade off": some subjects put more emphasis on accuracy, and respond slowly, some do the opposite.
A huge amount of work has been done on this problem, including the Inverse Efficiency Score referred to by @LGBogaardt. There are alternatives along the same lines - statistical models which take both accuracy and speed into account.
Another option, which I prefer, is to fit a sequential sampling model, such as a diffusion model (there are many other options here, but that paper is a good start). This basically models the process underlying your participants' responses as a simple, semi-random movement over time, which drifts generally towards the right answer, but, because of random noise, sometimes gives the wrong answer instead, and moves less consistently under harder conditions.
The great strength of this method is that it allows you to make more informed inferences about the underlying processes, rather than just identifying one condition as "faster".
