An experiment I conducted recently used a 2 (between participants) x 3 (within participants) design. That is, participants were randomly allocated to one of two conditions, and then completed three similar tasks each (in a counterbalanced order).
In each of these tasks, participants made binary choices (2AFC) in a number of trials, each of which had a normatively correct answer.
In every trial, participants were presented a distractor, which was assumed to bias responses towards one of the alternatives. The tasks differed only in presence and magnitude of this distractor (i.e. no distractor vs. distractor of small and large magnitude).
I would like to examine the error rates (deviations from the normatively correct answer) across these conditions. I hypothesize that the error rate will increase when a distractor is present, but will not increase further when the magnitude of the distractor is increased. Also, I expect this increase to differ between the between-subjects conditions. The latter interaction is the central interest.
From discussions here and from the literature (Dixon, 2008; Jaeger, 2008), I gather that logit mixed-models are the appropriate analysis method, and that, in R, the lme4 package is the tool of choice.
While I could compute some basic analyses (e.g. random intercept model, random effects ANCOVA) with lme4, I am stuck as to how to apply the models to the design in question -- I have the feeling that I am very much thinking in terms of HLMs, and have not yet quite understood the entirety of mixed effects models. Therefore, I would be very grateful for your help.
I have two basic questions:
In a first analysis, I would like to consider the error rates in only those trials in which participants were biased towards the wrong answer. The first model would therefore look only at trials in which the bias would point away from the correct answer.
If my observations were independent, i would probably use a model like this:
correct ~ condition + distractor + condition:distractor
... but obviously, they aren't: Observations are grouped within a task (with a constant presence of a distractor) and within participants. My question, then, is this: How do I add the random effects to reflect this?
(If I haven't lost you already :-) ) Would it be possible to include all trials (those where the bias would be into the direction of the wrong and of the correct answer), and include this difference (i.e. direction of the bias) as a trial-level predictor?
In my imagination of HLMs, such a predictor (at the level of the trial) would depend on on the magnitude of the distractor present (at the level of the block), which again would depend on the condition of the participant (plus, possibly, a unique factor for each participant).
The interactions would then emerge ›automatically‹ as cross-level interactions. How would such a model be specified in the ›flat‹ lme4 syntax? (Would such a model make sense at all?)
Ok, I hope all of this makes sense -- I will gladly elaborate otherwise. Again, I would be most grateful for any ideas and comments regarding this analysis, and would like to thank you for taking the time and trouble to respond.
Dixon, P. (2008). Models of accuracy in repeated-measures designs. Journal of Memory and Language, 59(4), 447-456. doi: 10.1016/j.jml.2007.11.004
Jaeger T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434-446. doi: 10.1016/j.jml.2007.11.007