# Post hoc test after ANOVA with repeated measures using R

I have performed a repeated measures ANOVA in R, as follows:

aov_velocity = aov(Velocity ~ Material + Error(Subject/(Material)), data=scrd)
summary(aov_velocity)

• What syntax in R can be used to perform a post hoc test after an ANOVA with repeated measures?
• Would Tukey's test with Bonferroni correction be appropriate? If so, how could this be done in R?
• see this related question on post hoc tests for repeated measures designs stats.stackexchange.com/questions/575/… – Jeromy Anglim Aug 11 '11 at 10:08
• About your 2nd point: Tukey's HSD already includes a "correction" for multiplicity (at the level of the test statistic, not the alpha level like in Bonferroni's method). So, there's no need to combine both. – chl Aug 11 '11 at 11:35
• @chl: so we don't need to correct the alpha level during the multiple pairwise comparisons in the case of Tukey's HSD ? – stan Sep 26 '11 at 3:46
• @stan No. (Note: Unplanned (post-hoc) tests should be performed after the ANOVA showed a significant result, especially if it concerns a confirmatory approach.) – chl Sep 26 '11 at 8:15

What you could do is specify the model with lme and then use glht from the multcomp package to do what you want. However, lme gives slightly different F-values than a standard ANOVA (see also my recent questions here).

lme_velocity = lme(Velocity ~ Material, data=scrd, random = ~1|Subject)
anova(lme_velocity)

require(multcomp)
summary(glht(lme_velocity, linfct=mcp(Material = "Tukey")), test = adjusted(type = "bonferroni"))


For other contrasts then bonferroni, see e.g., the book on multcomp from the authors of the package.

You may also want to see this post on the R-mailing list, and this blog post for specifying a repeated measures ANOVA in R.

However, as shown in this question from me I am not sure if this approachs is identical to an ANOVA. Furthermore, glht only reports z-values instead of the usual t or F values. This seems to be uncommon, too.

So far, I haven't encountered another way of doing this.

If you want to stick with the aov() function you can use the emmeans package which can handle aovlist (and many other) objects.

library("emmeans")
# set orthogonal contrasts
options(contrasts = c("contr.sum", "contr.poly"))

aov_velocity <- aov(Velocity ~ Material + Error(Subject / Material), data = scrd)


After creating an emmGrid object as follows

emm <- emmeans(aov_velocity, ~ Material)


it is very easy to get all (post hoc) pairwise comparisons using the pairs() function or any desired contrast using the contrast() function of the emmeans package. Multiple-testing adjustments can be achieved via the adjust argument of these functions:

pairs(emm)  # adjust argument not specified -> default p-value adjustment in this case is "tukey"


For more information on this I found the detailed emmeans vignettes and the documentation to be very helpful.

Also, you can find a complete (reproducible) example including a description on how to get the correct contrast weights in my answer here.

Note, however, that using a univariate model for the post hoc tests can result in anti-conservative p-values if sphericity is violated.

If sphericity is met then you can run a two-way ANOVA:

aov_velocity = aov(Velocity~Material+Subject, data=scrd)
posthoc      = TukeyHSD(aov_velocity, 'Material', conf.level=0.95).