I investigate the psychometric properties of an existing scale in a data set with N = 714. The structure originally assumed by the authors is the following, from which different index scores (corresponding to the higher order variables) can be derived.
(simplified without errors and corvariances) I wanted to use CFAs to test different models and draw conclusions as to whether the use of common index scores is useful. First, I started testing a model with only the three first order factors.
(I have used MLR as an estimator because the items are not multivariate normally distributed. ) here a part of the output:
chisq.scaled cfi.scaled rmsea.scaled aic
524.974 0.864 0.127 18633.493
Estimator ML Robust
Model Fit Test Statistic 591.553 524.974
Degrees of freedom 42 42
P-value (Chi-square) 0.000 0.000
Scaling correction factor 1.127
for the Yuan-Bentler correction (Mplus variant)
User model versus baseline model:
Comparative Fit Index (CFI) 0.870 0.864
Tucker-Lewis Index (TLI) 0.830 0.822
Robust Comparative Fit Index (CFI) 0.871
Robust Tucker-Lewis Index (TLI) 0.831
Number of free parameters 24 24
Akaike (AIC) 18633.493 18633.493
Bayesian (BIC) 18743.194 18743.194
Sample-size adjusted Bayesian (BIC) 18666.988 18666.988
Root Mean Square Error of Approximation:
RMSEA 0.135 0.127
90 Percent Confidence Interval 0.126 0.145 0.118 0.136
P-value RMSEA <= 0.05 0.000 0.000
Robust RMSEA 0.135
90 Percent Confidence Interval 0.125 0.145
Standardized Root Mean Square Residual:
SRMR 0.059 0.059
Parameter Estimates:
Information Observed
Observed information based on Hessian
Standard Errors Robust.huber.white
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
FA =~
A1 1.000 0.746 0.746
A2 0.890 0.053 16.863 0.000 0.664 0.664
A3 0.998 0.064 15.575 0.000 0.745 0.745
A4 0.934 0.045 20.666 0.000 0.696 0.697
A5 0.882 0.058 15.193 0.000 0.658 0.658
FB =~
B1 1.000 0.851 0.852
B2 0.983 0.042 23.171 0.000 0.836 0.837
B3 1.034 0.044 23.447 0.000 0.880 0.881
B4 0.784 0.034 23.084 0.000 0.668 0.668
B5 0.862 0.042 20.484 0.000 0.733 0.734
FC =~
C1 1.000 0.999 1.000
Covariances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
FA ~~
FB 0.304 0.034 9.007 0.000 0.479 0.479
FC 0.431 0.034 12.557 0.000 0.578 0.578
FB ~~
FC 0.391 0.039 9.898 0.000 0.459 0.459
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
A1 0.442 0.032 13.792 0.000 0.442 0.443
.A2 0.558 0.034 16.292 0.000 0.558 0.559
.A3 0.444 0.045 9.903 0.000 0.444 0.445
.A4 0.514 0.034 14.976 0.000 0.514 0.514
.A5 0.566 0.037 15.310 0.000 0.566 0.567
.B1 0.274 0.028 9.871 0.000 0.274 0.275
.B2 0.299 0.024 12.721 0.000 0.299 0.300
.B3 0.224 0.024 9.310 0.000 0.224 0.224
.B4 0.553 0.029 19.315 0.000 0.553 0.554
.B5 0.461 0.035 13.337 0.000 0.461 0.462
.C1 0.000 0.000 0.000
FA 0.556 0.052 10.632 0.000 1.000 1.000
FB 0.724 0.054 13.492 0.000 1.000 1.000
FC 0.999 0.048 20.620 0.000 1.000 1.000
The correlations between the items are neither too high nor too low and the loadings of all items were high. The internal consistencies of subscales A and B are also high. Since it is problematic that only one indicator is assigned to the third factor, I have also calculated it once with two factors and also once completely without the item, but it always comes out similarly bad results for model fit. What could be reasons for the bad fit? (In an EFA there was also no fundamentally different structure.)