0
$\begingroup$

I have conducted multiple mediation analysis (3 predictors+1control +3 mediators+1outcome)(all observed variables) using R-lavaan.

My full model, has bad RMSEA while the CFI seem ok (see outcome below).

I am wondering what can I do in order to improve my RMSEA to be RMSEA<0.08. 1)Shall I drop direct and indirect paths that are non significant, or is there another way?

2) I have 3 predictors(Py,Na,Ma), 1 control variable for the predictors (Ag), 3 mediators(Dm,Pr,Ld), and 1 outcome(Mo).

I want to use Ag as a control variable.

So I set the Ag path to 0 (in order to not predict anything)and I allowed it to covary with the predictors in order to control for that in my model. 2a) Is that the right way in order to control for something? Because when for example I delete completely this control variable(Ag), my other regressions/paths stay unchanged,and on top of that My RMSEA, when I deleted this control variable, become 000. 2b) Why might me this the case? How is it possible only 1 control variable to change RMSEA to 000?

note: In a multiple regression(lm()) I would just add Ag into my model in order to use it as a control for the predictors. However they have suggested to me that in SEM, I should allow Ag to covary with the predictors, and put its path to 0 in general, in order to make Ag a control variable (to the predictors)

My fit index including the one that I want as a control variable in the model:

 Number of observations                           333

  Estimator                                         ML
  Model Fit Test Statistic                      45.881
  Degrees of freedom                                 4
  P-value (Chi-square)                           0.000

Model test baseline model:

  Minimum Function Test Statistic              977.754
  Degrees of freedom                                28
  P-value                                        0.000

User model versus baseline model:

  Comparative Fit Index (CFI)                    0.956
  Tucker-Lewis Index (TLI)                       0.691

Loglikelihood and Information Criteria:

  Loglikelihood user model (H0)              -3310.110
  Loglikelihood unrestricted model (H1)      -3287.169

  Number of free parameters                         32
  Akaike (AIC)                                6684.219
  Bayesian (BIC)                              6806.080
  Sample-size adjusted Bayesian (BIC)         6704.574

Root Mean Square Error of Approximation:

  RMSEA                                          0.177
  90 Percent Confidence Interval          0.133  0.225
  P-value RMSEA <= 0.05                          0.000

Standardized Root Mean Square Residual:

  SRMR                                           0.036

Parameter Estimates:

  Information                                 Expected
  Information saturated (h1) model          Structured
  Standard Errors                             Standardode here

Here is without the control variable:

Number of observations                           333

  Estimator                                         ML
  Model Fit Test Statistic                       0.000
  Degrees of freedom                                 0
  Minimum Function Value               0.0000000000000

Model test baseline model:

  Minimum Function Test Statistic              824.389
  Degrees of freedom                                21
  P-value                                        0.000

User model versus baseline model:

  Comparative Fit Index (CFI)                    1.000
  Tucker-Lewis Index (TLI)                       1.000

Loglikelihood and Information Criteria:

  Loglikelihood user model (H0)              -2891.846
  Loglikelihood unrestricted model (H1)      -2891.846

  Number of free parameters                         28
  Akaike (AIC)                                5839.691
  Bayesian (BIC)                              5946.319
  Sample-size adjusted Bayesian (BIC)         5857.502

Root Mean Square Error of Approximation:

  RMSEA                                          0.000
  90 Percent Confidence Interval          0.000  0.000
  P-value RMSEA <= 0.05                             NA

Standardized Root Mean Square Residual:

  SRMR                                           0.000

Parameter Estimates:

  Information                                 Expected
  Information saturated (h1) model          Structured
  Standard Errors                             Standard

Here is my code for the full model:

    model.nodt<-'Mo_scaled~b1*Dm_scaled+b2*Pr_scaled+b3*Ld_scaled+c*Py_scaled+f*Na_scaled+h*Ma_scaled+0*Ag_scaled
Dm_scaled~a1*Py_scaled+d1*Na_scaled+g1*Ma_scaled
Pr_scaled~a2*Py_scaled+d2*Na_scaled+g2*Ma_scaled
Ld_scaled~a3*Py_scaled+d3*Na_scaled+g3*Ma_scaled



indirect1:=a1*b1
indirect2:=a2*b2
indirect3:=a3*b3
indirect4:=d1*b1
indirect5:=d2*b2
indirect6:=d3*b3
indirect7:=g1*b1
indirect8:=g2*b2
indirect9:=g3*b3



total:= f+c+h+(a1*b1) +(a2*b2)+(a3*b3)+(d1*b1)+(d2*b2)+(d3*b3)+(g1*b1)+(g2*b2)+(g3*b3)
Dm_scaled~~Pr_scaled
Pr_scaled~~Ld_scaled
Ld_scaled~~Dm_scaled
Na_scaled~~Py_scaled
Na_scaled~~Ma_scaled
Ma_scaled~~Py_scaled
Ag_scaled~~Py_scaled
Ag_scaled~~Na_scaled
Ag_scaled~~Ma_scaled
'
$\endgroup$
0
$\begingroup$

The reason RMSEA goes to zero when you include that predictor is that your model becomes fully saturated, meaning you are perfectly reproducing the covariance matrix of your variables just with directed path between some of them instead of covariances, which is what regression is. The fit statistics are not relevant to fully saturated models.

By setting certain paths to zero, you are saying that Ag has no direct covariance with the other variables, a claim that the fit statistics aim to test. Because this model constrained model has poor fit statistics, it's clearly a bad fit, and you should change those constraints.

In general, that is a bad constraint to impose. If you want to control for variable, you need to include it in a regression, not just have it covary with some variables. It needs to predict the outcome (i.e., the mediation predictor) in the model to adequately control for it. I don't know who this "they" is that told you not to do that. Your intuition with regression is correct. SEM with observed variables is just regression with additional steps, so the same rules apply.

Also in general, to uncover equation-wise fit problems, you can use model-implied instrumental variables. See Kirby & Bollen (2009) for the theory and check out the MIIVsem package to implement the methods, though this doesn't really apply to your problem.

$\endgroup$
  • $\begingroup$ Though when I include Ag as a predictor, with 0 path towards Mo, and allow it to covary with Na,Py,Ma(in order to control for), the RMS then become 0.17,CFI=.095. When I take Ag, complet out of my model, then the RMS becomes 00 So what you are telling me to do is to change that path from 0 to free to be estimated in order to control for that variable.So, I should use Ag as a predictor of the mediators(Dm,Pr,Ld) and the outcome(Mo). If you mean that; I did it, now is non saturated but my overall fit indexes of the model got significantly worst. CFI=O.O74, TLI= -3.166, RMSEA=2.110. $\endgroup$ – SeaBlue Jul 14 '18 at 8:50
  • $\begingroup$ What can I do now in order to improve my new model, as this is significantly worst than the first model. What should be the next steps? Maybe drop some direct/indirect paths that are non significant? or fixing some paths to 1 rather allow them to be estimated? $\endgroup$ – SeaBlue Jul 14 '18 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.