I tried to post my question in Stack Overflow but they suggest me to ask to the statistic expert (you), so here I am!
I have a mixed design with 1 between factor (Group: ASD, CTR) and three within factors (Time: pre, post; Trustee: good, bad; Step: 1,2,3,4,5).
I performed a mixed model anova (type III) with aov_car:
anova <- aov_car(rating ~ time*trustee*step*group + Error(id/time*trustee*step), data = datatemp, anova_table = list(es = "ges"))
I got a significant three way interaction: Time * Trustee * Step
I want to decompose the interaction by doing planned contrasts for time (I am just interested in the contrasts between pre and post for the different trustees and steps).
This is the code I used:
trustee_time_step <- emmeans(anova, ~ time | trustee | step) pairs(trustee_time_step)
results for comparison with time
By default, pairs() should perform a Tukey correction. However no correction is done on the contrasts.
The Tukey correction appears only if I use step to divide my contrasts:
step_time_trustee <- emmeans(anova, ~ step | trustee | time) pairs(step_time_trustee)
results for comparison with step
I guess it is because Time has just two levels (each contrast is just post vs. pre) while step has 5 levels (see "Tukey method for comparing a family of 5 estimates" at the end of the picture).
So this is my question: why pairs() does not apply correction in the first case? Even if post vs. pre is just one contrast, it is done 10 times (one for each combination of trustee and step) and thus it should be corrected in my opinion. What should I do? Shall I correct them manually? And if yes, how?
I am very confused about this matter, and I don't want to report uncorrected results if this is not "statistically good".
I hope my question is clear enough. Thank you for the help!