I ran a computer-based experiment in which there were two within-subject factors, A and B. So all participants got multiple trials in each A*B cell. There was also one between subject factor, C.

I'm trying to predict response time, so initially I did:

> lmer(rt ~ A*B*C + (1|subj)

but was told I should specify random effect interactions as well, e.g.:

> lmer(rt ~ A*B*C + (1|subj) + (1|A:subj) + (1|B:subj)

In that case, shouldn't I also specify the three-way interaction? e.g.:

> lmer(rt ~ A*B*C + (1|subj) + (1|A:subj) + (1|A:B:subj)

I understand the first model, but I'm not quite clear on the other two--though they all provide different results. Can someone clarify what these models do and which one is most appropriate?

  • $\begingroup$ (1|A:subj) is: a random intercept for subj nested in A, but I'm not quite sure if it is what you were interested in computing..? Check the documentation: cran.r-project.org/web/packages/lme4/vignettes/lmer.pdf $\endgroup$ – Tim Oct 26 '14 at 20:50
  • $\begingroup$ @Tim I don't know. What's typical, and is there a graphical depiction of what's going on in these models? Essentially, I'm interested in factoring out any variance due to individual differences, because I am only interested in the effects of the fixed factors. $\endgroup$ – Jeff Oct 27 '14 at 14:40
  • 1
    $\begingroup$ It would be probably best for you to read more on mixed models, e.g. amazon.com/Mixed-Effects-Models-S-PLUS-Statistics-Computing/dp/…, stat.columbia.edu/~gelman/arm, or amazon.co.uk/…, and in the lme4 vignette you'll find more information on defining formulas for lmer. $\endgroup$ – Tim Oct 27 '14 at 16:40

When you are specifying random effects in an lme4::lmer model, the random factors go on the left of the pipe and the non-independence grouping variables go on the right, so the fully specified model in your question would very likely be:

lmer(rt ~ A*B*C + (A*B|subj))

I took some time to explore the difference between a random effect on the left of the pipe to a random effect on the right side of the pipe but it made a better post on it's own than as an answer to your particular question.

RPubs doc

Gist code

The most noticeable difference between the following two models...

lmer(rt ~ A + (1|subj/A))
lmer(rt ~ A + (A|subj))

...is that the latter estimates a random correlation parameter between random intercepts and random slopes. If you remove that random correlation parameter...

lmer(rt ~ A + (1|subj/A))
lmer(rt ~ A + (1|subj) + (0+A|subj))

...the two models return the exact same fixed effects (parameter estimates and associated errors), though I would guess that similarity depends on the particular design of the study.

| cite | improve this answer | |
  • $\begingroup$ Thanks, this was very helpful and I appreciate your thoroughness! However your second model is different than mine because you only include the interaction term as a random effect: (1|subj:A) instead of (1|subj) + (1|subj:A) (or equivalently, (1|subj/A). Of the three models you specify, I feel like I understand the first and third models, but I'm still having trouble visualizing the second model. $\endgroup$ – Jeff Nov 1 '14 at 21:44
  • $\begingroup$ @Jeff - Pointed taken. I edited my response to acknowledge that I didn't match your situation perfectly. Sorry I didn't cover nested random effects here. $\endgroup$ – pedmiston Nov 1 '14 at 22:03
  • 2
    $\begingroup$ I hope a statistician can weigh in on this, but I believe a nested model (1|subj/A) is almost indistinguishable from a model without a random correlation parameter (1|subj) + (0+A|subj). At least, fitting those two models on the example dataset above yields identical error around the fixed effects. But I'm still not sure why you would want to nest subject in a factor of interest. $\endgroup$ – pedmiston Nov 1 '14 at 22:28
  • 2
    $\begingroup$ The logic, I suppose, is that (1|subj) accounts for the fact that each subject's mean deviates from the grand mean and (1|subj:A) accounts for the fact that effect of A differs by subject. Which sounds a lot like including a random slope. FWIW, i tried replacing (1|subj) + (1|A:subj) + (1|B:subj) in my model with (1|subj) + (0+A|subj) + (0+B|subj) and also got very similar results for my data set. $\endgroup$ – Jeff Nov 1 '14 at 22:47

With respect to the question which model is the most appropriate, I am still learning this for myself, but I can recommend an excellent paper covering this topic:

Barr, DJ, Levy R., Scheepers C., Tily HJ. (2013) Random effects structure for confirmatory hypothesis testing: Keep it maximal, Journal of Memory and Language 68 (2013) 255–278

As the title implies, the bottomline is that you should always try to include the maximum random effects structure, because only that guarantees that you do not inflate Type I error (it is also an excellent tutorial on what random intercepts and slopes do in the regression model).

However, sometimes complex random effects structures can lead to problems with model convergence. That is why some researchers propose a little less radical approach, and suggest that you should keep the maximal random effects structure that is justified by data (i.e. you test whether the random effects improve the model fit by using anova() and strip it to significant random effects only). Florian Jaeger covers this issue in his excellent blog:

http://hlplab.wordpress.com/2009/05/14/random-effect-structure/ http://hlplab.wordpress.com/2011/06/25/more-on-random-slopes/

| cite | improve this answer | |
  • 2
    $\begingroup$ Barr et al. paper is not a panacea. One needs to be a bit pragmatic trying to determine what level of model complexity your data will support. Over-parametrized models are not guaranteed to be reach an optimum and as a result subsequent (backwards) model selection procedure (like the anova() mentioned) are rendered invalid. $\endgroup$ – usεr11852 Nov 2 '14 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.