I have noticed a difference in post-hoc results produced from a repeated-measures ANOVA and a linear mixed model.
Here is reproducible code:
library(lme4) library(lmerTest) library(multcomp) library(agricolae) data(ham) #LINEAR MIXED MODEL WITH RESTRICTED MAXIMUM LIKELIHOOD model <- lmer(Informed.liking ~ Gender * Product + (1|Consumer), data=ham, REML=T) anova(model) #Gender SS = 3.854 #Gender MS = 3.8544 #Gender F = 0.8789 ham$Gender.Product <- interaction(ham$Gender, ham$Product) LMM.model.posthoc <- lmer(Informed.liking ~ Gender.Product + (1|Consumer), data=ham, REML=T) summary(glht(LMM.model.posthoc, linfct = mcp(Gender.Product = "Tukey")), test = adjusted("none")) #POST HOC COMPARISON OF 1.1 - 1.2: p = 0.191855 #POST HOC COMPARISON OF 1.2 - 1.3: p = 0.011224 #POST HOC COMPARISON OF 2.1 - 1.4: p = 0.671585 #POST HOC COPMARISON OF 1.3 - 2.2: p = 0.000209 #REPEATED MEASURES ANOVA summary(aov(Informed.liking ~ Gender * Product + Error(Consumer/Product), data=ham)) #Gender SS = 9.7 #Gender MS = 9.666 #Gender F = 0.879 anova <- aov(Informed.liking ~ Gender.Product, data=ham) summary(LSD.test(anova, "Gender.Product", group=F, console=TRUE, p.adj=c("none"))) #POST HOC COMPARISON OF 1.1 - 1.2: p = 0.2312 #POST HOC COMPARISON OF 1.2 - 1.3: p = 0.0202 #POST HOC COMPARISON OF 1.4 - 2.1: p = 0.6480 #POST HOC COPMARISON OF 1.3 - 2.2: p = 0.0001
The F-statistics are the same. However, the p-values from the post-hoc tests (Fisher's LSD in this example) differ, and I'm not sure why. I've noticed that SS and MS for Gender differ between the ANOVA and the LMM. As far as I can tell, I'm doing these analyses correctly.
In this example, is there a reason to prefer one method over the other? I would think these approaches should be equivalent here and I don't know why this isn't the case.
EDIT: This may be due to differences in p-value calculations for the packages. Post-hoc for the LMM uses glht from the Multcomp package, Post-hoc for ANOVA uses LSD.test from Agricolae. If this is the explanation, is there any reason to prefer one over the other in this situation? If not, what's the the explanation?
EDIT in response to design: In my situation, I have taken the same measurement on different parameters of individuals in different groups. Imagine I'm measuring blood concentration levels of different hormones from people of different ages.
In a repeated measures ANOVA, the parameter is the within-subjects effect (i.e. Hormone 1, Hormone 2, Hormone 3), and the groups are the between-subjects effect (i.e. Young, Middle-Age, Senior).
For a linear mixed model, the parameter and groups could be thought of as fixed effects, and individual as a random effect.
From my understanding, in this case both analyses would be equivalent. The differences in post-hoc p-values made me wonder if there's a difference in the approaches. I'm thinking now it's due to the differences in the packages and not in the analyses themselves, but I wanted to be sure.
Linear Mixed Model: concentration ~ parameter * group + (1|Individual)
Repeated Measures ANOVA: concentration ~ parameter * group + Error (Individual/Parameter)