When is a repeated measures ANOVA preferred over a mixed-effects model?

Question

In response to this question, regarding whether my design where I randomly presented participants with pictures from different categories was an example where I should use a repeated measures ANOVA, I got the answer that I should use a mixed-model instead, with one of the reasons being that I have two forms of dependencies: for subjects and for categories.

My question is now: Isn't it always the case that you have two dependencies in this way when doing this type of repeated measures design? That is, under what circumstances would a repeated-measures ANOVA be preferable to a mixed-effects modeling approach and why?

Answer 1

I'm not totally sure what actual model "repeated measures ANOVA" describes, but I think one general issue is whether to put random effects of any kind in a model rather than e.g. just adjust variance estimates to cover the induced dependencies (as in the Panel Corrected Standard Errors vs multilevel models debate in time series cross-sectional data analysis). So I'll have a go at that question first, then address yours.

Fixed and Random Effects

Two complementary principles about when to use a random rather than fixed effect are the following:

1. Represent a thing (subject, stimulus type, etc.) with a random effect when you are interested using the model to generalise to other instances of that thing not included in the current analysis, e.g. other subject or other stimulus types. If not use a fixed effect.
2. Represent a thing with a random effect when you think that for any instance of the thing, other instances in the data set are potentially informative about it. If you expect no such informativeness, then use a fixed effect.

Both motivate explicitly including subject random effects: you are usually interested in human populations in general and the elements of each subject's response set are correlated, predictable from each other and therefore informative about each other. It is less clear for things like stimuli. If there will only ever be three types of stimuli then 1. will motivate a fixed effect and 2. will make the decision depend on the nature of the stimuli.

Your questions

One reason to use a mixed model over a repeated effects ANOVA is that the former are considerably more general, e.g. they work equally easily with balanced and unbalanced designs and they are easily extended to multilevel models. In my (admittedly limited) reading on classical ANOVA and its extensions, mixed models seem to cover all the special cases that ANOVA extensions do. So I actually can't think of a statistical reason to prefer repeated measures ANOVA. Others may be able to help here. (A familiar sociological reason is that your field prefers to read about methods its older members learnt in graduate school, and a practical reason is that it might take a bit longer to learn how to use mixed models than a minor extension of ANOVA.)

Note

A caveat for using random effects, most relevant for non-experimental data, is that to maintain consistency you have to either assume that the random effects are uncorrelated with the model's fixed effects, or add fixed effect means as covariates for the random effect (discussed e.g. in Bafumi and Gelman's paper).

Answer 2

If your participants see the exact same pictures in each condition (which is obviously not the case in your original example because each category will presumably contain different pictures), an ANOVA on the cell means probably tells you exactly what you want to know. One reason to prefer it is that it's somewhat easier to understand and communicate (including to reviewers when you will try to publish your study).

But basically yes, if you run experiments where a number of people have to do something in response to a few conditions (e.g. pictures categories) with repeated trials in each condition, it's always the case that you have two sources of variability. Researchers in some fields (e.g. psycholinguistics) routinely use multilevel models (or some other older alternatives like Clark's F1/F2 analysis) precisely for that reason whereas other fields (e.g. a lot of work in mainstream experimental psychology) basically ignore the issue (for no other reason that being able to get away with it, from what I can tell).

This paper also discusses this question:

Raaijmakers, J.G.W., Schrijnemakers, J.M.C., & Gremmen, F. (1999). How to Deal with "The Language-as-Fixed-Effect Fallacy": Common Misconceptions and Alternative Solutions. Journal of Memory and Language, 41 (3), 416-426.

Answer 3

Never. A repeated measures ANOVA is one type, probably the simplest, of mixed effects model. I would recommend not even learning repeated measures except to know how to fit one as a mixed effects, but to learn mixed effects methods. It takes more effort as they can't be understood as recipes but are much more powerful as they can be expanded to multiple random effects, different correlation structures and handle missing data.

See Gueorguieva, R., & Krystal, J. H. (2011). Move over ANOVA. Arch Gen Psychiatry, 61, 310–317. http://doi.org/10.1001/archpsyc.61.3.310