# Evaluate different mediation analysis models in R

I have 3 variables which I used to conduct a mediation analysis. Since it is unclear to me whether which variable should be the (in)dependent variable and which should be mediator, I constructed 3 models. Using package mediation, I got the results which show all three models are statistically significant:

Model 1:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  54.0422     4.7155  11.460  < 2e-16 ***
X            27.5089     9.9512   2.764  0.00747 **
M             0.9369     0.4571   2.050  0.04455 *

Causal Mediation Analysis

Nonparametric Bootstrap Confidence Intervals with the Percentile Method

Estimate 95% CI Lower 95% CI Upper p-value
ACME             7.2197       0.8546        17.56   0.022 *
ADE             27.5089      11.1184        44.51  <2e-16 ***
Total Effect    34.7285      19.1319        53.34  <2e-16 ***
Prop. Mediated   0.2079       0.0292         0.54   0.022 *
---

Model 2:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  54.0422     4.7155  11.460  < 2e-16 ***
X             0.9369     0.4571   2.050  0.04455 *
M            27.5089     9.9512   2.764  0.00747 **
Causal Mediation Analysis

Nonparametric Bootstrap Confidence Intervals with the Percentile Method

Estimate 95% CI Lower 95% CI Upper p-value
ACME              0.447        0.141         0.78   0.002 **
ADE               0.937        0.153         1.89   0.014 *
Total Effect      1.384        0.544         2.37  <2e-16 ***
Prop. Mediated    0.323        0.114         0.80   0.002 **
---

Model 3:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.158881   0.096979  -1.638  0.10635
X            0.010813   0.005477   1.974  0.05274 .
M            0.003932   0.001423   2.764  0.00747 **

Causal Mediation Analysis

Nonparametric Bootstrap Confidence Intervals with the Percentile Method

Estimate 95% CI Lower 95% CI Upper p-value
ACME            5.44e-03     1.15e-03         0.01   0.006 **
ADE             1.08e-02    -7.11e-05         0.02   0.052 .
Total Effect    1.63e-02     7.09e-03         0.02   0.004 **
Prop. Mediated  3.35e-01     6.56e-02         0.97   0.010 **
---


By quickly looking at ACME and also the p value for M, I would say model 2 and 3 appear to be superior than model 1. Also, for model 3, the X-->Y path was significant before the mediation effect was accounted for and no longer significant after, which isn't the case for model 2. That tells me that perhaps model 3 explains the data the best. However, is that evaluation sufficient? So my question is whether there is a way to statistically evaluate/compare these three models and pick out the one that best fits the data. I thought I'd try lavaan but from what I can see I can use it to run models with multiple mediators so not what I'm trying to do here. Any advice?

Finally, a note on lavaan. You can use lavaan to test multiple mediators, and you can use it to test one mediator. What you cannot do is causal mediation, which includes an interaction between the treatment and the mediator and allows for variable types other than continuous for the mediator and outcome. mediation can do all of this. Also note that you cannot use fit statistics to test whether a given mediation model fits well because it will likely be a saturated model which always fits the data perfectly even if it makes no substantive sense.