I have the following data set where I am testing how fast people can recognise words in different conditions - pre-test (before learning), immediate (straight after learning) and delayed (a long time after learning).
I first inspect my mean RTs. Subjects are faster during the delayed condition and slowest during immediate
df %>% drop_na %>% dplyr::group_by(test.time) %>% dplyr::summarise(meanRT = mean(RT)) test.time meanRT <fct> <dbl> 1 delayed 1340. 2 immediate 1484. 3 pretest 1386.
I fit a linear mixed effect models to investigate the effect of test time on RTs and conduct post hoc contrasts using the emmeans package
library(lmerTest) libary(emmeans) mod.1 <- lmerTest::lmer(RT ~ test.time + (1 | Ppt.No), data = df %>% drop_na) emm1 = emmeans(mod.1, specs = pairwise ~ test.time) emm1 $emmeans test.time emmean SE df lower.CL upper.CL delayed 1343 82.9 43.2 1176 1510 immediate 1483 83.0 43.5 1315 1650 pretest 1402 83.0 43.3 1235 1569 Degrees-of-freedom method: kenward-roger Confidence level used: 0.95 $contrasts contrast estimate SE df t.ratio p.value delayed - immediate -140.0 36.2 2454 -3.865 0.0003 delayed - pretest -59.0 36.0 2455 -1.639 0.2295 immediate - pretest 80.9 36.3 2455 2.227 0.0668 Degrees-of-freedom method: kenward-roger P value adjustment: tukey method for comparing a family of 3 estimates
The contrasts are what I would expect by looking at my arithmetic means - delayed is significantly different from immediate, pretest is significantly different from immediate, and delayed and pretest are not significantly different.
However my estimated marginal means are vastly different from my arithmetic means - why is this? I have read on a few sites that it might be because my design is unbalanced? I investigated this and found that some participants have far fewer observations than others in my sample. Is this a valid reason for my estimated marginal means to be so different? And should I just stick to reporting arithmetic means when describing my post hoc contrasts?
Any help is appreciated, thank you!