1
$\begingroup$

After reading about interactions contrasts in emmeans, I just wanted to make sure I understood it correctly. Say I have a model with a group*time interaction effect, and I set up emmeans as follows:

emm <- emmeans(lme, ~ Group * Session)

And then use

contrast(emm, interaction = TRUE, "pairwise", adjust="mvt")

It outputs something like

Group_pairwise        Session_pairwise   estimate    SE     df   t.ratio p.value
Group_A - Group_B     Session1 - Session2  x.xxx       x.xxxx xx.x x.xxx   0.001
Group_A - Group_B     Session1 - Session3 ...
...

Does the first line for example then say that the difference between Session 1 and 2 within Group_A is significantly different to the difference between Session 1 and 2 within Group_B?

So it is basically a contrast of contrasts?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ Yes, that is correct, and interaction contrasts are indeed contrasts of contrasts. But the code looks wrong. Should be contrast(emm, interaction = "pairwise") $\endgroup$
    – Russ Lenth
    Commented Aug 22, 2022 at 0:30
  • $\begingroup$ Thanks. Just following up on this - how can I use eff_size to calculate cohens d and it's confidence interval for the interaction contrast? For instance, I'm aiming at a cohens d and CI for the abovementioned example. So basically the effect size for the group difference of the change scores (Session 1 - Session 2). $\endgroup$
    – Zucchini
    Commented Feb 15, 2023 at 21:44
  • $\begingroup$ When I write << interactioncontrast <- contrast(emm, interaction = "pairwise") >> and then << eff_size(interactioncontrast, sigma = sigma(lme), edf = df.residual(lme)) >> I'm not getting the desired effect size for << (A-B) - (Ses.1-Ses.2) >> but instead something like << (A - B Ses.1 - Ses.2) - (A - B Ses.1 - Ses.3) >> with cohens d and CI... $\endgroup$
    – Zucchini
    Commented Feb 15, 2023 at 21:44
  • $\begingroup$ (The reviewer specifically asked for cohens d and CI in this case ...) $\endgroup$
    – Zucchini
    Commented Feb 15, 2023 at 21:46
  • 1
    $\begingroup$ I honestly don't know, as I have never seen people report Cohen d effect sizes based on interaction contrasts. I guess it's possible to scale the interaction contrasts by dividing by sigma. But do you have an example of a published report where Cohen d effect sizes are reported for interaction contrasts? $\endgroup$
    – Russ Lenth
    Commented Feb 16, 2023 at 3:53

1 Answer 1

Reset to default
2
$\begingroup$

I guess if a Cohen's d makes sense, you can do something like

CON <- contrast(emm, interaction = TRUE, "pairwise", adjust="mvt")
eff_size(CON, sigma = ???, edf = ???, method = "identity")

(see the examples in the help page for eff_size) but you need to replace the ???s with reasonable values -- and be able to explain and defend them. You apparently have a mixed model, and I'm not sure Cohen's d is even defined because there are multiple sigma values involved. But I suppose that for edf you can use #groups - 1 for the coarsest grouping. For sigma, possibly you can do VarCorr(model) to see all the variance estimates; and then combine them. For instance, if there are three SDs sd1, sd2, sd3 you might use

sigma <- sqrt(sd1^2 + sd2^2 + sd3^2)

I am very leery of all of this because you can't find an example where this was actually ever done with interaction contrasts, and because in any but the simplest situations, the whole idea of effect size is flaky; people improvise something such as what I suggest, and then nobody really knows what you're talking about. You wind up with numbers, but they answer a question that can't be stated clearly.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ BTW, sometimes reviewers ask for things that can't be produced. You need to learn how to talk back to them and patiently explain why you can't comply. $\endgroup$
    – Russ Lenth
    Commented Feb 21, 2023 at 16:34
  • $\begingroup$ Thank you very much for your extensive replies! In fact I have already partly followed your advise: As the query sounded somewhat reasonable, I have computed the effect sizes (I simply calculated cohens by dividing by sigma), but only present those in the supplement and do not interet/mention them any further; while explaining the reviewer why. The reviewer seems to be happy with this compromise :) $\endgroup$
    – Zucchini
    Commented Feb 22, 2023 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.