I have a following problem:

I have some research data that need to be analysed. There are three randomized independent variables (binary) and one dependent variable (reaction time). The data consists of several thousand observations (repeated measures for about 30 subjects, with full randomization):

[distractor, IV1, IV2, RT]

I need to test this data for interactions. AFAIK, I should use some version of ANOVA test, but I don't know which one exactly is appropriate here. I'd like to use R, or perhaps Scipy; I read some tutorials and articles on this topic, also here on SE. Unfortunately, my understanding of statistical analysis is very limited, and it seems I don't have enough theoretical knowledge to decide how it should be done. I would be really grateful for a possibly detailed explanation and tips on solving this with R.

  • $\begingroup$ By repeated measures fully randomized, do you mean each subject saw all levels of the independent variables? Is there missing data? Is the data balanced (equal number of observations for each subject in each level of the IVs)? $\endgroup$
    – le_andrew
    Commented May 19, 2015 at 2:10
  • $\begingroup$ Yes, the data is balanced, and each subject went through all possible combinations of IVs, in random order. No missing data. $\endgroup$
    – machaerus
    Commented May 19, 2015 at 7:12

2 Answers 2


What you have described is a classic repeated measures ANOVA design. The data being balanced with no missing data makes the analysis even simpler (i.e. you could do it by hand in Excel in 5 minutes). You mentioned you randomized the order of the repeated measures. That means you cannot explicitly account for order effects (or at least it would be difficult to) but the randomization hopefully cancels out any systematic effect.

There are several ways to do repeated measures ANOVA in R. rnso provided the mixed effect regression method (it has like a billion other names like mixed models, multilevel models, etc.). Technically speaking, when you have balanced data with no missing values, it is equivalent to repeated measures ANOVA. Unless you plan on teaching yourself mixed effects regression on top of any other statistical training you may do, it is probably better to learn the more basic technique though. For starters, lmer will not give you p values. You may want to dive into that controversy and figure out why it doesn't and how to estimate them if you need them. Or you may just want to use an ANOVA command and avoid the headache. If you want to learn mixed effects regression, though, it is more flexible. I use it for all my repeated measures designs.

Here is good reference page for analyzing different kinds of repeated measures designs. It gives the analysis using aov lme and lmer. lme and lmer are both mixed effects regression functions from two different packages (rnso used lmer in his answer). Your exact design is not in there, though. You have a 3 way repeated measures design. They have 3 way split plot designs (an anachronistic word, unless of course you do agriculture research, for when you have both within subject factors and between subject factors... you only have within). However, it is easy to extend the 2-way repeated measures design to 3-way though. You simply add the third factor to both the fixed effects and the error strata. Your analysis is given by:

summary(aov(rt ~ distractor * IV1 * IV2 + Error(id/(distractor * IV1 * IV2)), data=dat))

Note that your data should be in long form (one row for each observation, meaning you'll have multiple rows for each subject). If your data is in wide format (i.e. one line per subject with multiple columns for different observations) you'll have to reshape your data or use another method (for instance the Anova function which I mention below.

There are other ANOVA commands in R that some people prefer, such as Anova from the car package or ezANOVA from the ez package. Regardless of what you do, though, you saying that you have very limited understanding of statistical analysis brings up some problems (that can't be fixed in a single forum reply). First, you mentioned you need to check for interactions. That single model above will tell you about the 3-way interaction, but the 2-way interaction terms may not mean what you think they mean, and their interpretation depends on how the IVs are coded. To check for both 2-way and 3-way interactions, you may have to run multiple models changing the coding to get what you want. Second, ANOVA is based on a set of assumptions that your data may or may not meet (normally distributed residuals, homogeneity of variance, and linearity). Repeated measures ANOVA adds additional assumptions (i.e. the assumptions of sphericity). Some functions, like ANOVA from the car package gives you tests of sphericity. Reaction time data rarely meets the normality assumption, but log transforming the reaction times often fixes the problem. Linearity just means that the data don't have non-linear patterns (or if they do you model that). If your IVs are dichotomous, this won't (can't) be a problem. Anyway, this is all stuff you can figure out pretty easily on Google. I mention them in passing here so you know that if you want to do the analysis right its a bit more complicated than the single line of code I put.


Mixed effects regression model may work for you. In R code:

lmer(RT ~ IV1 * IV2 * distractor + (1 | ID), data=mydata)

where ID is subject identification, since it is a repeated measures of 30 subjects.

(1 | ID) type syntax is used with lmer for varying intercept or slope models. See: http://jaredknowles.com/journal/2013/11/25/getting-started-with-mixed-effect-models-in-r

  • $\begingroup$ Thank you! Could you please explain, what is this part about: (1 | ID)? $\endgroup$
    – machaerus
    Commented May 19, 2015 at 7:16
  • $\begingroup$ Please see the edit with link. $\endgroup$
    – rnso
    Commented May 19, 2015 at 7:37
  • $\begingroup$ The answer provided won't test for interactions (which were mentioned in the question, though a very minor edit will (replace the first 2 plus signs with asterisks). $\endgroup$
    – le_andrew
    Commented May 19, 2015 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.