Independent variable becomes insignificant after adding control variable. Mediation is significant but doesnt make sense

Question

I am doing a linear regression analysis and have the problem as stated above. When only the independent variable (IV) and the dependent variable (DV) are included in my model, I get this:

  Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.58751    0.01147  51.202  < 2e-16 ***
IV       0.27696    0.06564   4.219 3.13e-05 ***

Adding a control variable (CV) to the regression, leads to this:

             Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.508030   0.092986  -5.464 8.92e-08 ***
IV      0.134419   0.057986   2.318    0.021 *  
CV        0.049427   0.004173  11.845  < 2e-16 ***

I checked the variance inflation factors, which are both 1.050193 (they are identical, does that mean anything?). So multicollinearity doesn't seem to be the problem.

Then I thought of Mediation (Baron & Kenny). Because the IV gets insignificant, when adding the CV, the CV seems to be the mediator. I checked this using the lavaan package:

Regressions:
                   Estimate  Std.Err  z-value  P(>|z|)
  Y ~                                                 
    X          (c)    0.134    0.058    2.328    0.020
  M ~                                                 
    X          (a)    3.038    0.725    4.191    0.000
  Y ~                                                 
    M          (b)    0.049    0.004   11.896    0.000

Variances:
                   Estimate  Std.Err  z-value  P(>|z|)
   .Y                 0.021    0.002   13.229    0.000
   .M                 3.468    0.262   13.229    0.000

Defined Parameters:
                   Estimate  Std.Err  z-value  P(>|z|)
    direct            0.134    0.058    2.328    0.020
    indirect          0.150    0.038    3.953    0.000
    total             0.285    0.067    4.262    0.000
    prop_mediated     0.528    0.128    4.118    0.000

Now to my problem: to me this doesn't make sense. To me I cannot construct an explanation for this result (X->M->Y).

Is there anything else I can check, a test I can do? The data is archival, so remeasuring, or measuring something like measuring X before M is not possible.

I am looking forward to your replies!

Answer 1

What you have described is mediation, right on the nose. So, you have it. Not only does the IV go from significant to not (which isn't really a good test of anything, as Andrew Gelman showed in an article called "the difference between `significant' and 'not significant' is not, itself, statistically significant" but the paraemeter estimate got a lot closer to 0. The mediation analysis you did confirms that it is mediation (there are some newer approaches to mediation, but that probably won't matter).

You haven't told us what the variables are, but just that it "doesn't make sense" for there to be mediation. Well, that might be very good. I mean, it could be that the data is wrong, or that you created errors bringing it from the archive, or something like that. But maybe you've discovered something new. That should thrill you.