0
$\begingroup$

I tried to post my question in Stack Overflow but they suggest me to ask to the statistic expert (you), so here I am!

I have a mixed design with 1 between factor (Group: ASD, CTR) and three within factors (Time: pre, post; Trustee: good, bad; Step: 1,2,3,4,5).

I performed a mixed model anova (type III) with aov_car:

anova <- aov_car(rating ~ time*trustee*step*group + Error(id/time*trustee*step), data = datatemp, anova_table = list(es = "ges"))

I got a significant three way interaction: Time * Trustee * Step

I want to decompose the interaction by doing planned contrasts for time (I am just interested in the contrasts between pre and post for the different trustees and steps).

This is the code I used:

trustee_time_step <- emmeans(anova, ~ time | trustee | step) 
pairs(trustee_time_step)

results for comparison with time

enter image description here

By default, pairs() should perform a Tukey correction. However no correction is done on the contrasts.

The Tukey correction appears only if I use step to divide my contrasts:

step_time_trustee <- emmeans(anova,  ~ step | trustee | time) 
pairs(step_time_trustee)

results for comparison with step

enter image description here

I guess it is because Time has just two levels (each contrast is just post vs. pre) while step has 5 levels (see "Tukey method for comparing a family of 5 estimates" at the end of the picture).

So this is my question: why pairs() does not apply correction in the first case? Even if post vs. pre is just one contrast, it is done 10 times (one for each combination of trustee and step) and thus it should be corrected in my opinion. What should I do? Shall I correct them manually? And if yes, how?

I am very confused about this matter, and I don't want to report uncorrected results if this is not "statistically good".

I hope my question is clear enough. Thank you for the help!

Improve this question
$\endgroup$
1
  • $\begingroup$ 1. Please correct all instances of Tuckey to Tukey (unless you really mean Tuckey, in which case please provide a reference) - you should check the spelling while you edit (e.g I am guessing 'cose' is probably meant to be 'code'). 2. If you inadvertently post to the wrong group the correct action is to flag for migration, not to repost. $\endgroup$
    – Glen_b
    Jan 17, 2019 at 4:56

1 Answer 1

Reset to default
2
$\begingroup$

The Tukey correction is applied separately to each “by” group, and with just one comparison, there is no multiplicity to correct for.

If you want to correct for multiplicity for that whole set of comparisons within groups, the Tukey correction is not appropriate, because that adjustment is based on the correlation structure for a set of pairwise comparisons of a single set of means. I suggest doing this:

summary(pairs(trustee_time_step),
    by = NULL, adjust = “mvt”)

This removes the by grouping AFTER the comparisons are computed, and applies the multivariate t correction to the set of results. This is the “exact” correction method provided the normality and homogeneous-variance assumptions are correct.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you a lot. This worked perfectly fine. I imagined there was some "theoretical mistake" in what I wanted to apply (Tukey)! $\endgroup$
    – claudia
    Jan 17, 2019 at 16:23
  • $\begingroup$ Off-topic: we have a tag lsmeans but do not have a tag [emmeans]. I think we should create one and make them synonyms. Question: which tag name do you think should be the "main" one? BTW, I have edited the wiki excerpt for [lsmeans] to say "Least-squares means, a.k.a. estimated marginal means, are predictions from a model over a regular grid, possibly averaged over some dimensions. Use this tag for R packages emmeans and lsmeans." -- let me know if you think it can be improved. $\endgroup$
    – amoeba
    Feb 5, 2019 at 15:15
  • $\begingroup$ I think “emmeans” but maybe that’s selfish because SAS has a large and earlier community. $\endgroup$
    – Russ Lenth
    Feb 5, 2019 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.