2
$\begingroup$

I have a sem model with non-normal distributed indicators measured with Likert-scale (7 points - agreement). I have 4 LVs and 1 observed (N=287, data collection still in progress).

As variables in my models are not normally distributed, I specified (before running the sem model) that the variables have to be ordered (I am using R):

df %<>%
mutate_at(.funs = list(~ordered(.)), 
.vars = vars(out_1, out_2, ...))

Then, I run the sem model and Lavaan automatically switched to a diagonally weighted estimator (DWLS). All my hypotheses have been confirmed, cfi is really high (0.986). When I worked with MLM/MLR estimators, with different datasets, honestly, it never happened that everything went so smoothly. So I am concerned that I am not able to read the sem output correctly and I would like to ask you:

1) from the output below, it there something (some indices) indicating that the model is doing something wrong (e.g, overfitting?);

2) what are the necessary checks in lavaan that I should carry out to see if the model is good?

3) so basically, can I trust this results?

My doubt is that the sample size is still too small. However, I am going to show these preliminary data soon, in the next days, and I would like to understand if I can trust these results.

> summary(fit, fit.measures= TRUE, ci= TRUE, standardized = TRUE) 
lavaan 0.6-4 ended normally after 45 iterations

  Optimization method                           NLMINB
  Number of free parameters                        118

  Number of observations                           287

  Estimator                                       DWLS      Robust
  Model Fit Test Statistic                     631.682     466.955
  Degrees of freedom                               114         114
  P-value (Chi-square)                           0.000       0.000
  Scaling correction factor                                  1.580
  Shift parameter                                           67.282
    for simple second-order correction (Mplus variant)

Model test baseline model:

  Minimum Function Test Statistic            69433.935   25525.694
  Degrees of freedom                               120         120
  P-value                                        0.000       0.000

User model versus baseline model:

  Comparative Fit Index (CFI)                    0.993       0.986
  Tucker-Lewis Index (TLI)                       0.992       0.985

  Robust Comparative Fit Index (CFI)                            NA
  Robust Tucker-Lewis Index (TLI)                               NA

Root Mean Square Error of Approximation:

  RMSEA                                          0.126       0.104
  90 Percent Confidence Interval          0.117  0.136       0.094  0.114
  P-value RMSEA <= 0.05                          0.000       0.000

  Robust RMSEA                                                  NA
  90 Percent Confidence Interval                                NA     NA

Standardized Root Mean Square Residual:

  SRMR                                           0.107       0.107

Parameter Estimates:

  Information                                 Expected
  Information saturated (h1) model        Unstructured
  Standard Errors                           Robust.sem

Latent Variables:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
  frn =~                                                                                  
    frn_1             0.963    0.005  175.060    0.000    0.952    0.973    1.024    0.967
    frn_2             0.932    0.008  113.337    0.000    0.916    0.949    0.992    0.940
    frn_3             0.908    0.010   87.003    0.000    0.888    0.929    0.966    0.918
    frn_4             0.902    0.012   75.140    0.000    0.878    0.925    0.959    0.912
    frn_5             0.823    0.018   44.801    0.000    0.787    0.859    0.876    0.839
  mpos =~                                                                                 
    mpos1             0.645    0.034   19.125    0.000    0.579    0.711    0.762    0.706
    mpos2             0.904    0.018   49.988    0.000    0.869    0.940    1.069    0.929
    mpos3             0.927    0.019   49.320    0.000    0.890    0.964    1.095    0.946
  mneg =~                                                                                 
    mneg1             0.580    0.037   15.729    0.000    0.507    0.652    0.586    0.584
    mneg2             0.838    0.031   27.192    0.000    0.777    0.898    0.847    0.841
    mneg3             0.855    0.033   25.542    0.000    0.790    0.921    0.865    0.858
  out =~                                                                                  
    out_1             0.815    0.032   25.869    0.000    0.753    0.877    1.044    0.976
    out_2             0.828    0.030   28.034    0.000    0.770    0.886    1.061    0.989
    out_3             0.729    0.028   25.898    0.000    0.674    0.784    0.934    0.884
    out_4             0.769    0.028   27.606    0.000    0.715    0.824    0.985    0.927
    out_5             0.583    0.029   19.836    0.000    0.525    0.640    0.747    0.720

Regressions:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
  out ~                                                                                   
    frn               0.210    0.072    2.922    0.003    0.069    0.351    0.174    0.174
    mpos              0.560    0.067    8.402    0.000    0.429    0.690    0.516    0.516
    mneg             -0.257    0.070   -3.653    0.000   -0.394   -0.119   -0.203   -0.203
  frn ~                                                                                   
    nomy              0.244    0.041    5.967    0.000    0.164    0.324    0.229    0.341
  mpos ~                                                                                  
    nomy              0.422    0.043    9.737    0.000    0.337    0.507    0.358    0.532
  mneg ~                                                                                  
    nomy             -0.101    0.042   -2.381    0.017   -0.183   -0.018   -0.099   -0.148

Intercepts:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
   .frn_1             0.000                               0.000    0.000    0.000    0.000
   .frn_2             0.000                               0.000    0.000    0.000    0.000
   .frn_3             0.000                               0.000    0.000    0.000    0.000
   .frn_4             0.000                               0.000    0.000    0.000    0.000
   .frn_5             0.000                               0.000    0.000    0.000    0.000
   .mpos1             0.000                               0.000    0.000    0.000    0.000
   .mpos2             0.000                               0.000    0.000    0.000    0.000
   .mpos3             0.000                               0.000    0.000    0.000    0.000
   .mneg1             0.000                               0.000    0.000    0.000    0.000
   .mneg2             0.000                               0.000    0.000    0.000    0.000
   .mneg3             0.000                               0.000    0.000    0.000    0.000
   .out_1             0.000                               0.000    0.000    0.000    0.000
   .out_2             0.000                               0.000    0.000    0.000    0.000
   .out_3             0.000                               0.000    0.000    0.000    0.000
   .out_4             0.000                               0.000    0.000    0.000    0.000
   .out_5             0.000                               0.000    0.000    0.000    0.000
   .frn               0.000                               0.000    0.000    0.000    0.000
   .mpos              0.000                               0.000    0.000    0.000    0.000
   .mneg              0.000                               0.000    0.000    0.000    0.000
   .out               0.000                               0.000    0.000    0.000    0.000

Thresholds:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
    frn_1|t1         -0.246    0.211   -1.165    0.244   -0.660    0.168   -0.246   -0.232
    frn_1|t2          0.495    0.204    2.431    0.015    0.096    0.895    0.495    0.468
    frn_1|t3          0.860    0.205    4.195    0.000    0.458    1.262    0.860    0.812
    frn_1|t4          1.571    0.212    7.413    0.000    1.156    1.986    1.571    1.483
    frn_1|t5          2.008    0.222    9.040    0.000    1.572    2.443    2.008    1.895
    frn_1|t6          2.776    0.245   11.311    0.000    2.295    3.258    2.776    2.621
    frn_2|t1         -0.250    0.204   -1.226    0.220   -0.649    0.149   -0.250   -0.236
    frn_2|t2          0.461    0.199    2.313    0.021    0.070    0.851    0.461    0.436
    frn_2|t3          0.909    0.201    4.530    0.000    0.516    1.302    0.909    0.861
    frn_2|t4          1.679    0.209    8.029    0.000    1.270    2.089    1.679    1.591
    frn_2|t5          2.228    0.220   10.147    0.000    1.798    2.659    2.228    2.111
    frn_2|t6          2.658    0.239   11.121    0.000    2.190    3.126    2.658    2.518
    frn_3|t1         -0.253    0.210   -1.206    0.228   -0.666    0.159   -0.253   -0.241
    frn_3|t2          0.331    0.203    1.632    0.103   -0.066    0.728    0.331    0.314
    frn_3|t3          0.777    0.202    3.847    0.000    0.381    1.173    0.777    0.738
    frn_3|t4          1.358    0.206    6.577    0.000    0.953    1.762    1.358    1.289
    frn_3|t5          1.833    0.211    8.695    0.000    1.419    2.246    1.833    1.740
    frn_3|t6          2.460    0.231   10.659    0.000    2.007    2.912    2.460    2.336
    frn_4|t1         -0.263    0.203   -1.293    0.196   -0.661    0.136   -0.263   -0.250
    frn_4|t2          0.395    0.195    2.030    0.042    0.014    0.777    0.395    0.376
    frn_4|t3          0.800    0.193    4.135    0.000    0.421    1.178    0.800    0.760
    frn_4|t4          1.395    0.198    7.057    0.000    1.008    1.783    1.395    1.326
    frn_4|t5          1.817    0.202    9.015    0.000    1.422    2.212    1.817    1.726
    frn_4|t6          2.333    0.207   11.287    0.000    1.928    2.738    2.333    2.217
    frn_5|t1         -0.474    0.217   -2.190    0.029   -0.899   -0.050   -0.474   -0.454
    frn_5|t2          0.010    0.202    0.049    0.961   -0.386    0.406    0.010    0.009
    frn_5|t3          0.465    0.199    2.332    0.020    0.074    0.855    0.465    0.445
    frn_5|t4          1.395    0.205    6.806    0.000    0.993    1.797    1.395    1.336
    frn_5|t5          1.830    0.212    8.647    0.000    1.415    2.245    1.830    1.753
    frn_5|t6          2.371    0.222   10.700    0.000    1.936    2.805    2.371    2.271
    mpos1|t1         -0.815    0.251   -3.253    0.001   -1.307   -0.324   -0.815   -0.756
    mpos1|t2         -0.313    0.204   -1.538    0.124   -0.713    0.086   -0.313   -0.290
    mpos1|t3          0.292    0.181    1.613    0.107   -0.063    0.648    0.292    0.271
    mpos1|t4          0.895    0.175    5.122    0.000    0.553    1.238    0.895    0.829
    mpos1|t5          1.661    0.175    9.479    0.000    1.318    2.005    1.661    1.539
    mpos1|t6          2.621    0.183   14.343    0.000    2.263    2.979    2.621    2.429
    mpos2|t1         -0.892    0.246   -3.629    0.000   -1.375   -0.410   -0.892   -0.776
    mpos2|t2         -0.085    0.183   -0.464    0.643   -0.444    0.274   -0.085   -0.074
    mpos2|t3          0.464    0.167    2.787    0.005    0.138    0.790    0.464    0.403
    mpos2|t4          1.178    0.163    7.215    0.000    0.858    1.498    1.178    1.024
    mpos2|t5          1.958    0.164   11.971    0.000    1.638    2.279    1.958    1.702
    mpos2|t6          2.740    0.169   16.172    0.000    2.408    3.073    2.740    2.382
    mpos3|t1         -0.532    0.226   -2.347    0.019   -0.975   -0.088   -0.532   -0.459
    mpos3|t2          0.068    0.183    0.372    0.710   -0.290    0.427    0.068    0.059
    mpos3|t3          0.653    0.171    3.828    0.000    0.319    0.987    0.653    0.564
    mpos3|t4          1.392    0.167    8.313    0.000    1.064    1.720    1.392    1.202
    mpos3|t5          1.960    0.170   11.560    0.000    1.628    2.293    1.960    1.693
    mpos3|t6          2.863    0.166   17.235    0.000    2.538    3.189    2.863    2.473
    mneg1|t1         -0.749    0.198   -3.791    0.000   -1.136   -0.362   -0.749   -0.746
    mneg1|t2         -0.191    0.191   -1.001    0.317   -0.565    0.183   -0.191   -0.190
    mneg1|t3          0.261    0.189    1.383    0.167   -0.109    0.631    0.261    0.260
    mneg1|t4          0.845    0.192    4.401    0.000    0.468    1.221    0.845    0.841
    mneg1|t5          1.623    0.211    7.684    0.000    1.209    2.037    1.623    1.617
    mneg1|t6          2.170    0.237    9.166    0.000    1.706    2.634    2.170    2.162
    mneg2|t1         -1.825    0.222   -8.228    0.000   -2.260   -1.391   -1.825   -1.811
    mneg2|t2         -1.123    0.196   -5.728    0.000   -1.508   -0.739   -1.123   -1.114
    mneg2|t3         -0.669    0.192   -3.488    0.000   -1.046   -0.293   -0.669   -0.664
    mneg2|t4         -0.061    0.193   -0.315    0.753   -0.440    0.318   -0.061   -0.060
    mneg2|t5          0.785    0.210    3.731    0.000    0.373    1.198    0.785    0.779
    mneg2|t6          1.560    0.276    5.659    0.000    1.020    2.101    1.560    1.548
    mneg3|t1         -1.374    0.202   -6.786    0.000   -1.770   -0.977   -1.374   -1.362
    mneg3|t2         -0.765    0.192   -3.979    0.000   -1.142   -0.388   -0.765   -0.759
    mneg3|t3         -0.300    0.190   -1.577    0.115   -0.673    0.073   -0.300   -0.298
    mneg3|t4          0.284    0.195    1.458    0.145   -0.098    0.667    0.284    0.282
    mneg3|t5          1.004    0.219    4.586    0.000    0.575    1.433    1.004    0.996
    mneg3|t6          1.532    0.253    6.055    0.000    1.036    2.027    1.532    1.519
    out_1|t1         -1.455    0.332   -4.390    0.000   -2.105   -0.806   -1.455   -1.360
    out_1|t2         -0.924    0.236   -3.912    0.000   -1.386   -0.461   -0.924   -0.863
    out_1|t3         -0.353    0.208   -1.698    0.089   -0.761    0.054   -0.353   -0.330
    out_1|t4          0.144    0.212    0.678    0.498   -0.272    0.559    0.144    0.134
    out_1|t5          0.619    0.217    2.850    0.004    0.193    1.045    0.619    0.579
    out_1|t6          1.214    0.226    5.364    0.000    0.770    1.658    1.214    1.135
    out_2|t1         -0.931    0.256   -3.635    0.000   -1.434   -0.429   -0.931   -0.869
    out_2|t2         -0.858    0.248   -3.467    0.001   -1.344   -0.373   -0.858   -0.801
    out_2|t3         -0.319    0.207   -1.540    0.124   -0.724    0.087   -0.319   -0.297
    out_2|t4          0.057    0.200    0.285    0.776   -0.334    0.448    0.057    0.053
    out_2|t5          0.632    0.203    3.116    0.002    0.235    1.030    0.632    0.590
    out_2|t6          1.268    0.209    6.081    0.000    0.859    1.677    1.268    1.183
    out_3|t1         -0.974    0.255   -3.814    0.000   -1.475   -0.474   -0.974   -0.922
    out_3|t2         -0.759    0.233   -3.261    0.001   -1.216   -0.303   -0.759   -0.719
    out_3|t3         -0.304    0.207   -1.466    0.143   -0.710    0.102   -0.304   -0.288
    out_3|t4          0.251    0.202    1.239    0.215   -0.146    0.647    0.251    0.237
    out_3|t5          0.700    0.207    3.378    0.001    0.294    1.107    0.700    0.663
    out_3|t6          1.122    0.211    5.307    0.000    0.708    1.536    1.122    1.062
    out_4|t1         -1.274    0.273   -4.669    0.000   -1.809   -0.739   -1.274   -1.199
    out_4|t2         -1.017    0.235   -4.330    0.000   -1.477   -0.557   -1.017   -0.957
    out_4|t3         -0.524    0.201   -2.603    0.009   -0.918   -0.129   -0.524   -0.493
    out_4|t4         -0.119    0.198   -0.598    0.550   -0.507    0.270   -0.119   -0.112
    out_4|t5          0.333    0.196    1.695    0.090   -0.052    0.718    0.333    0.313
    out_4|t6          0.929    0.201    4.624    0.000    0.535    1.323    0.929    0.875
    out_5|t1         -1.639    0.399   -4.109    0.000   -2.421   -0.857   -1.639   -1.581
    out_5|t2         -1.238    0.345   -3.588    0.000   -1.914   -0.562   -1.238   -1.194
    out_5|t3         -0.821    0.319   -2.572    0.010   -1.446   -0.195   -0.821   -0.792
    out_5|t4         -0.673    0.308   -2.189    0.029   -1.277   -0.070   -0.673   -0.650
    out_5|t5         -0.492    0.298   -1.653    0.098   -1.076    0.091   -0.492   -0.475
    out_5|t6         -0.151    0.289   -0.523    0.601   -0.718    0.416   -0.151   -0.146

Variances:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
   .frn_1             0.073                               0.073    0.073    0.073    0.065
   .frn_2             0.130                               0.130    0.130    0.130    0.117
   .frn_3             0.175                               0.175    0.175    0.175    0.158
   .frn_4             0.187                               0.187    0.187    0.187    0.169
   .frn_5             0.322                               0.322    0.322    0.322    0.296
   .mpos1             0.584                               0.584    0.584    0.584    0.501
   .mpos2             0.182                               0.182    0.182    0.182    0.137
   .mpos3             0.140                               0.140    0.140    0.140    0.105
   .mneg1             0.664                               0.664    0.664    0.664    0.659
   .mneg2             0.298                               0.298    0.298    0.298    0.293
   .mneg3             0.269                               0.269    0.269    0.269    0.264
   .out_1             0.055                               0.055    0.055    0.055    0.048
   .out_2             0.024                               0.024    0.024    0.024    0.021
   .out_3             0.243                               0.243    0.243    0.243    0.218
   .out_4             0.158                               0.158    0.158    0.158    0.140
   .out_5             0.517                               0.517    0.517    0.517    0.481
   .frn               1.000                               1.000    1.000    0.884    0.884
   .mpos              1.000                               1.000    1.000    0.717    0.717
   .mneg              1.000                               1.000    1.000    0.978    0.978
   .out               1.000                               1.000    1.000    0.609    0.609

Scales y*:
                   Estimate  Std.Err  z-value  P(>|z|) ci.lower ci.upper   Std.lv  Std.all
    frn_1             1.000                               1.000    1.000    1.000    1.000
    frn_2             1.000                               1.000    1.000    1.000    1.000
    frn_3             1.000                               1.000    1.000    1.000    1.000
    frn_4             1.000                               1.000    1.000    1.000    1.000
    frn_5             1.000                               1.000    1.000    1.000    1.000
    mpos1             1.000                               1.000    1.000    1.000    1.000
    mpos2             1.000                               1.000    1.000    1.000    1.000
    mpos3             1.000                               1.000    1.000    1.000    1.000
    mneg1             1.000                               1.000    1.000    1.000    1.000
    mneg2             1.000                               1.000    1.000    1.000    1.000
    mneg3             1.000                               1.000    1.000    1.000    1.000
    out_1             1.000                               1.000    1.000    1.000    1.000
    out_2             1.000                               1.000    1.000    1.000    1.000
    out_3             1.000                               1.000    1.000    1.000    1.000
    out_4             1.000                               1.000    1.000    1.000    1.000
    out_5             1.000                               1.000    1.000    1.000    1.000

Thank you very much!

$\endgroup$

1 Answer 1

2
$\begingroup$

RMSEA and SRMR are pretty high.

That's often a sign that you have strong relationships in your data, and so your null chi-square is high. That makes the incremental fit indices (CFI, etc) look good. You can see this by the way the loadings are so high (especially on frn).

I don't understand what you mean by "All my hypotheses have been confirmed". The chi-square test was highly significant. You don't confirm (null) hypotheses, you reject them, or you fail to reject them.

Your sample size is a bit on the low side but probably not too bad.

I would take a look at your modification indices and see if anything stands out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.