I have performed a repeated measures ANOVA in R, as follows:

aov_velocity = aov(Velocity ~ Material + Error(Subject/(Material)), data=scrd)
  • What syntax in R can be used to perform a post hoc test after an ANOVA with repeated measures?
  • Would Tukey's test with Bonferroni correction be appropriate? If so, how could this be done in R?
  • 1
    $\begingroup$ see this related question on post hoc tests for repeated measures designs stats.stackexchange.com/questions/575/… $\endgroup$ – Jeromy Anglim Aug 11 '11 at 10:08
  • 3
    $\begingroup$ About your 2nd point: Tukey's HSD already includes a "correction" for multiplicity (at the level of the test statistic, not the alpha level like in Bonferroni's method). So, there's no need to combine both. $\endgroup$ – chl Aug 11 '11 at 11:35
  • 1
    $\begingroup$ @chl: so we don't need to correct the alpha level during the multiple pairwise comparisons in the case of Tukey's HSD ? $\endgroup$ – stan Sep 26 '11 at 3:46
  • 3
    $\begingroup$ @stan No. (Note: Unplanned (post-hoc) tests should be performed after the ANOVA showed a significant result, especially if it concerns a confirmatory approach.) $\endgroup$ – chl Sep 26 '11 at 8:15

What you could do is specify the model with lme and then use glht from the multcomp package to do what you want. However, lme gives slightly different F-values than a standard ANOVA (see also my recent questions here).

lme_velocity = lme(Velocity ~ Material, data=scrd, random = ~1|Subject)

summary(glht(lme_velocity, linfct=mcp(Material = "Tukey")), test = adjusted(type = "bonferroni"))

For other contrasts then bonferroni, see e.g., the book on multcomp from the authors of the package.

You may also want to see this post on the R-mailing list, and this blog post for specifying a repeated measures ANOVA in R.

However, as shown in this question from me I am not sure if this approachs is identical to an ANOVA. Furthermore, glht only reports z-values instead of the usual t or F values. This seems to be uncommon, too.

So far, I haven't encountered another way of doing this.


If sphericity is met then you can run a two-way ANOVA:

aov_velocity = aov(Velocity~Material+Subject, data=scrd)
posthoc      = TukeyHSD(aov_velocity, 'Material', conf.level=0.95).

If you want to stick with the aov() function you can use the emmeans package which can handle aovlist (and many other) objects.

# set orthogonal contrasts
options(contrasts = c("contr.sum", "contr.poly"))

aov_velocity <- aov(Velocity ~ Material + Error(Subject / Material), data = scrd)

After creating an emmGrid object as follows

emm <- emmeans(aov_velocity, ~ Material)

it is very easy to get all (post hoc) pairwise comparisons using the pairs() function or any desired contrast using the contrast() function of the emmeans package. Multiple-testing adjustments can be achieved via the adjust argument of these functions:

pairs(emm)  # adjust argument not specified -> default p-value adjustment in this case is "tukey"  

For more information on this I found the detailed emmeans vignettes and the documentation to be very helpful.

Also, you can find a complete (reproducible) example including a description on how to get the correct contrast weights in my answer here.

Note, however, that using a univariate model for the post hoc tests can result in anti-conservative p-values if sphericity is violated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.