I have a sem model with non-normal distributed indicators measured with Likert-scale (7 points - agreement). I have 4 LVs and 1 observed (N=287, data collection still in progress).
As variables in my models are not normally distributed, I specified (before running the sem model) that the variables have to be ordered (I am using R):
df %<>%
mutate_at(.funs = list(~ordered(.)),
.vars = vars(out_1, out_2, ...))
Then, I run the sem model and Lavaan automatically switched to a diagonally weighted estimator (DWLS). All my hypotheses have been confirmed, cfi is really high (0.986). When I worked with MLM/MLR estimators, with different datasets, honestly, it never happened that everything went so smoothly. So I am concerned that I am not able to read the sem output correctly and I would like to ask you:
1) from the output below, it there something (some indices) indicating that the model is doing something wrong (e.g, overfitting?);
2) what are the necessary checks in lavaan that I should carry out to see if the model is good?
3) so basically, can I trust this results?
My doubt is that the sample size is still too small. However, I am going to show these preliminary data soon, in the next days, and I would like to understand if I can trust these results.
> summary(fit, fit.measures= TRUE, ci= TRUE, standardized = TRUE)
lavaan 0.6-4 ended normally after 45 iterations
Optimization method NLMINB
Number of free parameters 118
Number of observations 287
Estimator DWLS Robust
Model Fit Test Statistic 631.682 466.955
Degrees of freedom 114 114
P-value (Chi-square) 0.000 0.000
Scaling correction factor 1.580
Shift parameter 67.282
for simple second-order correction (Mplus variant)
Model test baseline model:
Minimum Function Test Statistic 69433.935 25525.694
Degrees of freedom 120 120
P-value 0.000 0.000
User model versus baseline model:
Comparative Fit Index (CFI) 0.993 0.986
Tucker-Lewis Index (TLI) 0.992 0.985
Robust Comparative Fit Index (CFI) NA
Robust Tucker-Lewis Index (TLI) NA
Root Mean Square Error of Approximation:
RMSEA 0.126 0.104
90 Percent Confidence Interval 0.117 0.136 0.094 0.114
P-value RMSEA <= 0.05 0.000 0.000
Robust RMSEA NA
90 Percent Confidence Interval NA NA
Standardized Root Mean Square Residual:
SRMR 0.107 0.107
Parameter Estimates:
Information Expected
Information saturated (h1) model Unstructured
Standard Errors Robust.sem
Latent Variables:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
frn =~
frn_1 0.963 0.005 175.060 0.000 0.952 0.973 1.024 0.967
frn_2 0.932 0.008 113.337 0.000 0.916 0.949 0.992 0.940
frn_3 0.908 0.010 87.003 0.000 0.888 0.929 0.966 0.918
frn_4 0.902 0.012 75.140 0.000 0.878 0.925 0.959 0.912
frn_5 0.823 0.018 44.801 0.000 0.787 0.859 0.876 0.839
mpos =~
mpos1 0.645 0.034 19.125 0.000 0.579 0.711 0.762 0.706
mpos2 0.904 0.018 49.988 0.000 0.869 0.940 1.069 0.929
mpos3 0.927 0.019 49.320 0.000 0.890 0.964 1.095 0.946
mneg =~
mneg1 0.580 0.037 15.729 0.000 0.507 0.652 0.586 0.584
mneg2 0.838 0.031 27.192 0.000 0.777 0.898 0.847 0.841
mneg3 0.855 0.033 25.542 0.000 0.790 0.921 0.865 0.858
out =~
out_1 0.815 0.032 25.869 0.000 0.753 0.877 1.044 0.976
out_2 0.828 0.030 28.034 0.000 0.770 0.886 1.061 0.989
out_3 0.729 0.028 25.898 0.000 0.674 0.784 0.934 0.884
out_4 0.769 0.028 27.606 0.000 0.715 0.824 0.985 0.927
out_5 0.583 0.029 19.836 0.000 0.525 0.640 0.747 0.720
Regressions:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
out ~
frn 0.210 0.072 2.922 0.003 0.069 0.351 0.174 0.174
mpos 0.560 0.067 8.402 0.000 0.429 0.690 0.516 0.516
mneg -0.257 0.070 -3.653 0.000 -0.394 -0.119 -0.203 -0.203
frn ~
nomy 0.244 0.041 5.967 0.000 0.164 0.324 0.229 0.341
mpos ~
nomy 0.422 0.043 9.737 0.000 0.337 0.507 0.358 0.532
mneg ~
nomy -0.101 0.042 -2.381 0.017 -0.183 -0.018 -0.099 -0.148
Intercepts:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
.frn_1 0.000 0.000 0.000 0.000 0.000
.frn_2 0.000 0.000 0.000 0.000 0.000
.frn_3 0.000 0.000 0.000 0.000 0.000
.frn_4 0.000 0.000 0.000 0.000 0.000
.frn_5 0.000 0.000 0.000 0.000 0.000
.mpos1 0.000 0.000 0.000 0.000 0.000
.mpos2 0.000 0.000 0.000 0.000 0.000
.mpos3 0.000 0.000 0.000 0.000 0.000
.mneg1 0.000 0.000 0.000 0.000 0.000
.mneg2 0.000 0.000 0.000 0.000 0.000
.mneg3 0.000 0.000 0.000 0.000 0.000
.out_1 0.000 0.000 0.000 0.000 0.000
.out_2 0.000 0.000 0.000 0.000 0.000
.out_3 0.000 0.000 0.000 0.000 0.000
.out_4 0.000 0.000 0.000 0.000 0.000
.out_5 0.000 0.000 0.000 0.000 0.000
.frn 0.000 0.000 0.000 0.000 0.000
.mpos 0.000 0.000 0.000 0.000 0.000
.mneg 0.000 0.000 0.000 0.000 0.000
.out 0.000 0.000 0.000 0.000 0.000
Thresholds:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
frn_1|t1 -0.246 0.211 -1.165 0.244 -0.660 0.168 -0.246 -0.232
frn_1|t2 0.495 0.204 2.431 0.015 0.096 0.895 0.495 0.468
frn_1|t3 0.860 0.205 4.195 0.000 0.458 1.262 0.860 0.812
frn_1|t4 1.571 0.212 7.413 0.000 1.156 1.986 1.571 1.483
frn_1|t5 2.008 0.222 9.040 0.000 1.572 2.443 2.008 1.895
frn_1|t6 2.776 0.245 11.311 0.000 2.295 3.258 2.776 2.621
frn_2|t1 -0.250 0.204 -1.226 0.220 -0.649 0.149 -0.250 -0.236
frn_2|t2 0.461 0.199 2.313 0.021 0.070 0.851 0.461 0.436
frn_2|t3 0.909 0.201 4.530 0.000 0.516 1.302 0.909 0.861
frn_2|t4 1.679 0.209 8.029 0.000 1.270 2.089 1.679 1.591
frn_2|t5 2.228 0.220 10.147 0.000 1.798 2.659 2.228 2.111
frn_2|t6 2.658 0.239 11.121 0.000 2.190 3.126 2.658 2.518
frn_3|t1 -0.253 0.210 -1.206 0.228 -0.666 0.159 -0.253 -0.241
frn_3|t2 0.331 0.203 1.632 0.103 -0.066 0.728 0.331 0.314
frn_3|t3 0.777 0.202 3.847 0.000 0.381 1.173 0.777 0.738
frn_3|t4 1.358 0.206 6.577 0.000 0.953 1.762 1.358 1.289
frn_3|t5 1.833 0.211 8.695 0.000 1.419 2.246 1.833 1.740
frn_3|t6 2.460 0.231 10.659 0.000 2.007 2.912 2.460 2.336
frn_4|t1 -0.263 0.203 -1.293 0.196 -0.661 0.136 -0.263 -0.250
frn_4|t2 0.395 0.195 2.030 0.042 0.014 0.777 0.395 0.376
frn_4|t3 0.800 0.193 4.135 0.000 0.421 1.178 0.800 0.760
frn_4|t4 1.395 0.198 7.057 0.000 1.008 1.783 1.395 1.326
frn_4|t5 1.817 0.202 9.015 0.000 1.422 2.212 1.817 1.726
frn_4|t6 2.333 0.207 11.287 0.000 1.928 2.738 2.333 2.217
frn_5|t1 -0.474 0.217 -2.190 0.029 -0.899 -0.050 -0.474 -0.454
frn_5|t2 0.010 0.202 0.049 0.961 -0.386 0.406 0.010 0.009
frn_5|t3 0.465 0.199 2.332 0.020 0.074 0.855 0.465 0.445
frn_5|t4 1.395 0.205 6.806 0.000 0.993 1.797 1.395 1.336
frn_5|t5 1.830 0.212 8.647 0.000 1.415 2.245 1.830 1.753
frn_5|t6 2.371 0.222 10.700 0.000 1.936 2.805 2.371 2.271
mpos1|t1 -0.815 0.251 -3.253 0.001 -1.307 -0.324 -0.815 -0.756
mpos1|t2 -0.313 0.204 -1.538 0.124 -0.713 0.086 -0.313 -0.290
mpos1|t3 0.292 0.181 1.613 0.107 -0.063 0.648 0.292 0.271
mpos1|t4 0.895 0.175 5.122 0.000 0.553 1.238 0.895 0.829
mpos1|t5 1.661 0.175 9.479 0.000 1.318 2.005 1.661 1.539
mpos1|t6 2.621 0.183 14.343 0.000 2.263 2.979 2.621 2.429
mpos2|t1 -0.892 0.246 -3.629 0.000 -1.375 -0.410 -0.892 -0.776
mpos2|t2 -0.085 0.183 -0.464 0.643 -0.444 0.274 -0.085 -0.074
mpos2|t3 0.464 0.167 2.787 0.005 0.138 0.790 0.464 0.403
mpos2|t4 1.178 0.163 7.215 0.000 0.858 1.498 1.178 1.024
mpos2|t5 1.958 0.164 11.971 0.000 1.638 2.279 1.958 1.702
mpos2|t6 2.740 0.169 16.172 0.000 2.408 3.073 2.740 2.382
mpos3|t1 -0.532 0.226 -2.347 0.019 -0.975 -0.088 -0.532 -0.459
mpos3|t2 0.068 0.183 0.372 0.710 -0.290 0.427 0.068 0.059
mpos3|t3 0.653 0.171 3.828 0.000 0.319 0.987 0.653 0.564
mpos3|t4 1.392 0.167 8.313 0.000 1.064 1.720 1.392 1.202
mpos3|t5 1.960 0.170 11.560 0.000 1.628 2.293 1.960 1.693
mpos3|t6 2.863 0.166 17.235 0.000 2.538 3.189 2.863 2.473
mneg1|t1 -0.749 0.198 -3.791 0.000 -1.136 -0.362 -0.749 -0.746
mneg1|t2 -0.191 0.191 -1.001 0.317 -0.565 0.183 -0.191 -0.190
mneg1|t3 0.261 0.189 1.383 0.167 -0.109 0.631 0.261 0.260
mneg1|t4 0.845 0.192 4.401 0.000 0.468 1.221 0.845 0.841
mneg1|t5 1.623 0.211 7.684 0.000 1.209 2.037 1.623 1.617
mneg1|t6 2.170 0.237 9.166 0.000 1.706 2.634 2.170 2.162
mneg2|t1 -1.825 0.222 -8.228 0.000 -2.260 -1.391 -1.825 -1.811
mneg2|t2 -1.123 0.196 -5.728 0.000 -1.508 -0.739 -1.123 -1.114
mneg2|t3 -0.669 0.192 -3.488 0.000 -1.046 -0.293 -0.669 -0.664
mneg2|t4 -0.061 0.193 -0.315 0.753 -0.440 0.318 -0.061 -0.060
mneg2|t5 0.785 0.210 3.731 0.000 0.373 1.198 0.785 0.779
mneg2|t6 1.560 0.276 5.659 0.000 1.020 2.101 1.560 1.548
mneg3|t1 -1.374 0.202 -6.786 0.000 -1.770 -0.977 -1.374 -1.362
mneg3|t2 -0.765 0.192 -3.979 0.000 -1.142 -0.388 -0.765 -0.759
mneg3|t3 -0.300 0.190 -1.577 0.115 -0.673 0.073 -0.300 -0.298
mneg3|t4 0.284 0.195 1.458 0.145 -0.098 0.667 0.284 0.282
mneg3|t5 1.004 0.219 4.586 0.000 0.575 1.433 1.004 0.996
mneg3|t6 1.532 0.253 6.055 0.000 1.036 2.027 1.532 1.519
out_1|t1 -1.455 0.332 -4.390 0.000 -2.105 -0.806 -1.455 -1.360
out_1|t2 -0.924 0.236 -3.912 0.000 -1.386 -0.461 -0.924 -0.863
out_1|t3 -0.353 0.208 -1.698 0.089 -0.761 0.054 -0.353 -0.330
out_1|t4 0.144 0.212 0.678 0.498 -0.272 0.559 0.144 0.134
out_1|t5 0.619 0.217 2.850 0.004 0.193 1.045 0.619 0.579
out_1|t6 1.214 0.226 5.364 0.000 0.770 1.658 1.214 1.135
out_2|t1 -0.931 0.256 -3.635 0.000 -1.434 -0.429 -0.931 -0.869
out_2|t2 -0.858 0.248 -3.467 0.001 -1.344 -0.373 -0.858 -0.801
out_2|t3 -0.319 0.207 -1.540 0.124 -0.724 0.087 -0.319 -0.297
out_2|t4 0.057 0.200 0.285 0.776 -0.334 0.448 0.057 0.053
out_2|t5 0.632 0.203 3.116 0.002 0.235 1.030 0.632 0.590
out_2|t6 1.268 0.209 6.081 0.000 0.859 1.677 1.268 1.183
out_3|t1 -0.974 0.255 -3.814 0.000 -1.475 -0.474 -0.974 -0.922
out_3|t2 -0.759 0.233 -3.261 0.001 -1.216 -0.303 -0.759 -0.719
out_3|t3 -0.304 0.207 -1.466 0.143 -0.710 0.102 -0.304 -0.288
out_3|t4 0.251 0.202 1.239 0.215 -0.146 0.647 0.251 0.237
out_3|t5 0.700 0.207 3.378 0.001 0.294 1.107 0.700 0.663
out_3|t6 1.122 0.211 5.307 0.000 0.708 1.536 1.122 1.062
out_4|t1 -1.274 0.273 -4.669 0.000 -1.809 -0.739 -1.274 -1.199
out_4|t2 -1.017 0.235 -4.330 0.000 -1.477 -0.557 -1.017 -0.957
out_4|t3 -0.524 0.201 -2.603 0.009 -0.918 -0.129 -0.524 -0.493
out_4|t4 -0.119 0.198 -0.598 0.550 -0.507 0.270 -0.119 -0.112
out_4|t5 0.333 0.196 1.695 0.090 -0.052 0.718 0.333 0.313
out_4|t6 0.929 0.201 4.624 0.000 0.535 1.323 0.929 0.875
out_5|t1 -1.639 0.399 -4.109 0.000 -2.421 -0.857 -1.639 -1.581
out_5|t2 -1.238 0.345 -3.588 0.000 -1.914 -0.562 -1.238 -1.194
out_5|t3 -0.821 0.319 -2.572 0.010 -1.446 -0.195 -0.821 -0.792
out_5|t4 -0.673 0.308 -2.189 0.029 -1.277 -0.070 -0.673 -0.650
out_5|t5 -0.492 0.298 -1.653 0.098 -1.076 0.091 -0.492 -0.475
out_5|t6 -0.151 0.289 -0.523 0.601 -0.718 0.416 -0.151 -0.146
Variances:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
.frn_1 0.073 0.073 0.073 0.073 0.065
.frn_2 0.130 0.130 0.130 0.130 0.117
.frn_3 0.175 0.175 0.175 0.175 0.158
.frn_4 0.187 0.187 0.187 0.187 0.169
.frn_5 0.322 0.322 0.322 0.322 0.296
.mpos1 0.584 0.584 0.584 0.584 0.501
.mpos2 0.182 0.182 0.182 0.182 0.137
.mpos3 0.140 0.140 0.140 0.140 0.105
.mneg1 0.664 0.664 0.664 0.664 0.659
.mneg2 0.298 0.298 0.298 0.298 0.293
.mneg3 0.269 0.269 0.269 0.269 0.264
.out_1 0.055 0.055 0.055 0.055 0.048
.out_2 0.024 0.024 0.024 0.024 0.021
.out_3 0.243 0.243 0.243 0.243 0.218
.out_4 0.158 0.158 0.158 0.158 0.140
.out_5 0.517 0.517 0.517 0.517 0.481
.frn 1.000 1.000 1.000 0.884 0.884
.mpos 1.000 1.000 1.000 0.717 0.717
.mneg 1.000 1.000 1.000 0.978 0.978
.out 1.000 1.000 1.000 0.609 0.609
Scales y*:
Estimate Std.Err z-value P(>|z|) ci.lower ci.upper Std.lv Std.all
frn_1 1.000 1.000 1.000 1.000 1.000
frn_2 1.000 1.000 1.000 1.000 1.000
frn_3 1.000 1.000 1.000 1.000 1.000
frn_4 1.000 1.000 1.000 1.000 1.000
frn_5 1.000 1.000 1.000 1.000 1.000
mpos1 1.000 1.000 1.000 1.000 1.000
mpos2 1.000 1.000 1.000 1.000 1.000
mpos3 1.000 1.000 1.000 1.000 1.000
mneg1 1.000 1.000 1.000 1.000 1.000
mneg2 1.000 1.000 1.000 1.000 1.000
mneg3 1.000 1.000 1.000 1.000 1.000
out_1 1.000 1.000 1.000 1.000 1.000
out_2 1.000 1.000 1.000 1.000 1.000
out_3 1.000 1.000 1.000 1.000 1.000
out_4 1.000 1.000 1.000 1.000 1.000
out_5 1.000 1.000 1.000 1.000 1.000
Thank you very much!
