# Why do mediation and moderation analysis give substantively different results?

I have a study where I pair two people up and have them play a behavioral game together. I measure some change score before and after the game. I create a four-level factor variable called treatment that has the following values: MM, MF, FM, FF which describe your biological sex, and the biological sex of your partner.

Let's say I also have some index moderator_idx where I believe, for people in the MF and FM category, high levels of moderator_idx will be associated with high levels of the outcome. So I model the following:

> m1 <- lm_robust(outcome ~ treatment / moderator_idx -1,
cluster = team_id,
se = "stata",
data = data_full)

Estimate  Std. Error   t value    Pr(>|t|)      CI Lower     CI Upper  DF
treatmentFF                 9.6698108  4.31703853  2.239918 0.025776158   1.176747161 18.162874409 323
treatmentFM                -2.4975363  6.76639155 -0.369109 0.712288286 -15.809299377 10.814226686 323
treatmentMF                -6.5241575  5.42255272 -1.203152 0.229798459 -17.192138536  4.143823553 323
treatmentMM               -20.0332461 13.66149199 -1.466403 0.143511539 -46.909985777  6.843493496 323
treatmentFF:moderator_idx  -0.1041088  0.05135792 -2.027123 0.043470833  -0.205147088 -0.003070541 323
treatmentFM:moderator_idx   0.1580438  0.07965394  1.984131 0.048087366   0.001337787  0.314749868 323
treatmentMF:moderator_idx   0.1845383  0.06667904  2.767562 0.005973407   0.053358306  0.315718388 323
treatmentMM:moderator_idx   0.2405057  0.15315809  1.570310 0.117322055  -0.060807677  0.541819060 323


Examining only the interaction effects (e.g. treatmentFM:moderator_idx), I get a result that I suspected: for people in different-sex conditions (FM, MF), the effect of an increase in the moderator is associated with significant increases in the outcome of interest.

However, I can re-cast this analysis as a mediation analysis like so (where different_sex is a dummy variable set to 1 if you're paired with someone in the opposite sex, and sex is your own biological sex). Note that this is equivalent to a four-level treatment factor above...

med.fit <- lm(moderator_idx ~ different_sex * sex, data = data_full)
out.fit <- lm(outgroup_feelings_diff ~ sex * different_sex * moderator_idx, data = data_full)
med.out <- mediation::mediate(med.fit, out.fit, treat = "different_sex", mediator = "moderator_idx", robustSE = TRUE, sims = 1000)
summary(med.out)

Quasi-Bayesian Confidence Intervals

Estimate 95% CI Lower 95% CI Upper p-value
ACME (control)            -0.0484      -0.5168         0.37    0.84
ACME (treated)            -0.6666      -1.7546         0.20    0.14
ADE (control)             11.2765       7.7503        14.58  <2e-16 ***
ADE (treated)             10.6583       7.2103        13.88  <2e-16 ***
Total Effect              10.6099       7.2444        13.87  <2e-16 ***
Prop. Mediated (control)  -0.0032      -0.0525         0.04    0.84
Prop. Mediated (treated)  -0.0620      -0.1759         0.02    0.14
ACME (average)            -0.3575      -0.9851         0.13    0.15
ADE (average)             10.9674       7.5714        14.19  <2e-16 ***
Prop. Mediated (average)  -0.0326      -0.1000         0.01    0.15
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


And I get that ACME of the treated is non-significant. My question is: How can I get in the first analysis large and significant effects of moderation, but non-significant effects of mediation. What are the substantive differences between the two results, and which should I trust?

To be clear: I understand that mediation and moderation analyses are fundamentally different. I'm wondering the circumstances under which moderation and mediation analysis would produce effects in different directions (the effect of the moderator is positive for the FM and MF conditions in the moderation analysis, but it's negative in the mediation analysis).