# Significant pairwise comparisons from emtrends but marginal means are not-significant?

I'm quite new to this type of statistical analysis, so apologies if the question comes across as silly. I'm currently fitting a mixed lmer with two categorical (condition and outcome; each with 2 levels) and one continious predictor. I found a three-way interaction between them that I wanted to explore with the emmeans package. However, I'm not sure how to interpret the results from the emtrends function.

Basically, the contrasts between my categorical predictors are significant, but the marginal means for the trait continious predictor are not. Why would that be? Could an intepretation be that the effect is too small to be picked up on an individual condition-outcome basis, but still enough to show in the pairwise comparisons? Or does it have something to do with the levels of the trait predictor the function picks as default?

Here's the code I used for the emtrends and the resulting table. (Note: I've enabled the lmerTest.limit argument since I have more than 3000 observations; I'm also using the mvt adjustment for multiple comparisons).

trait_em <- emtrends(m2, pairwise ~ condition*outcome_rec, var = "trait",lmerTest.limit = 85999,adjust = "mvt")

$emtrends condition outcome_rec trait.trend SE df lower.CL upper.CL nonpain loss 0.148 0.144 97.9 -0.175 0.471 pain loss -0.156 0.147 97.4 -0.485 0.174 nonpain win -0.131 0.175 96.8 -0.525 0.262 pain win 0.148 0.170 97.0 -0.235 0.530 Degrees-of-freedom method: satterthwaite Confidence level used: 0.95 Conf-level adjustment: mvt method for 4 estimates$contrasts
contrast                    estimate     SE    df lower.CL upper.CL
nonpain loss - pain loss    0.303755 0.0611 113.4    0.157    0.451
nonpain loss - nonpain win  0.279664 0.3088  96.9   -0.467    1.026
nonpain loss - pain win     0.000653 0.3098  96.4   -0.749    0.750
pain loss - nonpain win    -0.024091 0.3179  96.3   -0.792    0.744
pain loss - pain win       -0.303102 0.3087  96.9   -1.048    0.442
nonpain win - pain win     -0.279011 0.0603 107.5   -0.425   -0.133

Degrees-of-freedom method: satterthwaite
Confidence level used: 0.95
Conf-level adjustment: mvt method for 6 estimates
$$$$


As you know, the emtrends part of the output shows whether the continuous predictor's slopes for each categorical x categorical combination differ from zero and the contrast part shows whether they differ from each other. Now, for instance nonpain-loss slope is positive but small and the pain-loss slope is negative but small. They are both too small to be considered to be reliably different from zero. However, they have different signs (one is positive and one is negative), and thus their difference (absolute difference of ~.30) is large enough to produce a significant difference between the slopes. The same is happening with nonpain win - pain win slopes.

The emtrends function does not pick any default values of the continuous predictor in this context. It estimates the regression slope between the continuous predictor variable and the outcome as usual for each cat x cat combination.

EDIT. you may want to modify your emmeans code to only extract the contrasts of interest (I'm not sure in your case but in a comparable situation I wouldn't be interested in comparing nonpain win to pain loss etc., I'd just like to see pain vs no pain within win and pain vs no pain within loss or vice versa)

Like this

trait_em <- emtrends(m2, pairwise ~ condition|outcome_rec, var="trait"...)


This way your output is more readable. It seems you can say that pain vs. no pain moderates the trait effect on outcome differently within loss vs. win conditions (you could also unpack the interaction the other way around: that loss vs. win moderates the trait effect differently within pain than within no pain, if that makes more sense).

Visualize the result by plotting the trait slope for pain and non-pain conditions and make different plots for win and loss, I think that will get you started on the interpretation. You get a ready plot using emmip

emmip(m2, condition ~ trait|outcome_rec, cov.reduce=range) #you need the cov.reduce argument to make emmip to understand the trait variable as continuous or something like that.
#to get different plots for pain and no pain and have win and loss as the "legend" variable, just swap their places in the code
`
• Thank you, this does clarify things a bit! As I mentioned, I'm new to this type of analysis, so does this mean the interaction I'm observing is purely due to the fact that the slopes are negative in some cases but positive in others? Essentially, how would I go interpreting these results? Nov 22, 2023 at 10:48
• Interpretation depends on context, but just as with any interaction model, a non-significant main effect does not mean interaction effect can't be real (this extends to : 2-way interaction doesn't need to be significant for a 3-way interaction to be real). So, I'd interpret your results saying the relation between continuous predictor and outcome is statistically significantly different between the conditions specified in contrasts. Nov 22, 2023 at 10:54
• In sum, based on your stats there's no reason to think the interaction is just due to technicalities or noise. Visualize it and interpret using your substance knowledge. Nov 22, 2023 at 11:04
• And what makes marginal means appropriate here? Nov 22, 2023 at 12:55