1
$\begingroup$

I would like to ask a question regarding a post-hoc analysis using R package emmeans. I did a LME model analysis of a study of 2 groups x 4 measurement sessions. Now I am using emmeans for post-hoc comparisons. I am interested in finding out if the values within each of the groups differ between the sessions and also if both groups differ from each other at any session. In emmeans, I used

emm1 <– emmeans(model, ~ Session | Group)

pairs(emm1, adjust = "mvt")

as well as

emm2 <– emmeans(model, ~ Group | Session)

pairs(emm1, adjust = "mvt")

For Session|Group it outputs a comparison of the sessions, for both groups separately. P values are corrected for 6 comparisons. For Group|Session it outputs a comparison of both groups, separately for each session. Therefore, values are not corrected for multiple comparisons (as each session only consists of two means to compare, so the argument adjust is ignored). This normal behaviour is also illustrated on the vignette page for emmeans.

However, regarding my aim of the post-hoc analysis: should I force a multiple comparison correction? For example with the following:

summary(pairs(emm2), by = NULL, adjust = "mvt")

To me, applying a correction for 4 comparisons (groups at each session) sounds like it makes sense. What are your thoughts? And if so, would I have to use the same argument (with by=NULL) for emm1, too? This would combine comparisons for both groups (12 tests). While it seems rational to treat both emm1 and emm2 equal from a mathematical point of view, I don't think it is necessary from a logical point of view...

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ I think you should be consistent. If you do separate adjustments of session comparisons for each group, then you should do separate adjustments comparisons of groups for each session, even though the latter is no adjustment. $\endgroup$
    – Russ Lenth
    Commented Aug 20, 2022 at 2:45
  • $\begingroup$ Thanks! Any thoughts on whether it is better to force adjustments (use by=NULL) for both of them, or leave both of them as they are with fewer comparisons to correct for? (From a scientific/ logical point of view) $\endgroup$
    – Zucchini
    Commented Aug 20, 2022 at 20:58
  • 1
    $\begingroup$ Depends on your discipline. But Id probably do separate adjustments for each group. Main thing is to clearly explain what you did. $\endgroup$
    – Russ Lenth
    Commented Aug 21, 2022 at 0:32
  • 1
    $\begingroup$ You can use rbind and/or [] to combine or subset results and get all of the comparisons you want. $\endgroup$
    – Russ Lenth
    Commented Aug 22, 2022 at 0:26
  • 1
    $\begingroup$ My preference is to treat each group as a separate family, which is what happens by default. $\endgroup$
    – Russ Lenth
    Commented Aug 23, 2022 at 21:01

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.