Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am analyzing data from an experiment in which participants had to respond to Yes/No-questions of two types as quickly as possible. Hence, there are two dependent variables: response time and correctness of the response. There are 80 responses for every participant in each of the conditions. My hypothesis is that one type of question leads to slower or less correct responses.

Can anybody think of a reason why the following analysis might be flawed?

I determined, for every participant, the probability of giving a correct response before a certain point in time, i.e., $P(RT < t_n \wedge response~correct)$. As far as I can see, this corresponds to the ECDF multiplied by the proportion of correct responses. I removed all data points that equal 0 and fit a linear mixed-effects model with the following specification (in lme4):

$logit(P_{corrRT}) \sim time + condition + time:condition + (1|participant)$

As far as I can see, a main effect of condition or an interaction between condition and time would mean that there are differences in either RT or accuracy. Am I making any incorrect implicit assumptions, or does this analysis make sense?

share|improve this question
up vote 4 down vote accepted

The first thing I can see wrong with it is that glmer handles the working out of the probability of correct RT for you.

With accuracy the score of 1 for correct and 0 for incorrect...

lmer( acc ~ time * condition + (1|part), family = 'binomial')

This would be a much better way to do it. You could look at predicted values from the model with RT held at a constant (say, the mean) to see what accuracies would be like given a fixed RT.

One issue you'll have here is that the acc ~ time relationship isn't going to be linear in the logistic space. Even with persistent stimuli the accuracy falls off at later RTs... often you see a dip at some point, say 1200, or 1500 ms. You might deal with this by making time quadratic. Or you might get rid of longer RTs. If the RTs aren't to the initial presentation of the stimulus anymore maybe they don't measure what you want?

share|improve this answer
+1, but a minor point is that using lmer() with family='binomial' gives a warning in the latest version of lme4. The note in the release guide: "In the past, lmer automatically called glmer when family was specified. It still does so, but now warns the user that they should preferably use glmer directly." – smillig Oct 14 '13 at 13:54

There's a more recent paper which gives a tutorial-like introduction and examines the underlying model and fit a bit:

Davidson, D. J. und A. E. Martin (2013). Modeling accuracy as a function of response time with the generalized linear mixed effects model. Acta Psychologica, 144:83– 96.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.