I've fitted a one-factor model to data originating from a unidimensional 8-item scale; sample size is 400. Because the scale uses a Likert response format, and data does not follow a multivariate normal distribution, I have used DWLS. I have gotten non-significant results for Chi-squared, and therefore extremely good values for the rest of the fit indexes I have calculated. Since this is the first time I am seeing these type of results, I wonder: is it possible? Am I missing something? Should I instead use robust estimation methods (WLSMV)?

lavaan 0.6-5 ended normally after 29 iterations

  Estimator                                       DWLS
  Optimization method                           NLMINB
  Number of free parameters                         16

  Number of observations                           400

Model Test User Model:

  Test statistic                                16.916
  Degrees of freedom                                20
  P-value (Chi-square)                           0.658

Model Test Baseline Model:

  Test statistic                              4000.700
  Degrees of freedom                                28
  P-value                                        0.000

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    1.000
  Tucker-Lewis Index (TLI)                       1.001

Clearly it's possible, because it happened.

Typically when you have very low chi-squares, you sometimes have very lower CFI/TLI - that was the first think I looked at, because they indicate lower power.

You don't have low power, you just have a well fitting model. This is not a problem.

  • $\begingroup$ Thanks! I am so used to report a significant Chi-squared, disregard it, and then move on to other fit indexes to justify the model that I needed someone else to look at these results just to be sure :) $\endgroup$ – Bea Hs Dec 23 '19 at 19:02
  • $\begingroup$ If you can add the correlation matrix, I'd be interested to see it. $\endgroup$ – Jeremy Miles Dec 23 '19 at 19:08
  • 1
    $\begingroup$ It appears that you have not declared the observed variables to be ordinal, but are treating the Likert responses as if they were continuous. It is possible that you have extremely flat (platykurtic) distributions, which are suppressing the DWLS chi-square. Honestly, the results look like what I get when I simulate data for a classroom assignment. But you did say that the set of items were unidimensional, and if that is so, then these results are what you should expect. Your sample size is probably large enough for asymptotic properties to hold. $\endgroup$ – Ed Rigdon Dec 23 '19 at 22:32
  • $\begingroup$ @Ed Rigdon - nice spot! $\endgroup$ – Jeremy Miles Dec 23 '19 at 23:14
  • 1
    $\begingroup$ @EdRigdon Hi, I just saw your comment. You were right regarding not having declared the observed variables as ordered. I just did and I got a significant chi-square. The one-factor model still achieves an almost perfect fit to the data: CFI/TLI are now 0.99, RMSEA and SRMR are below typical threshold levels etc. so the end result is the same but I am happy I can report the correct chi-square and p-value. Thanks! $\endgroup$ – Bea Hs Jan 11 at 9:39

Note that it is very common to have a non-significant Chi-squared fit statistic in CFA testing, as it is heavily sensitive to large sample sizes and higher model complexity (i.e. a number of indicators in your model). In your case, it is very unsurprising that the Chi-squared in non-significant, but both CFI and TLI are large. I will gently echo @JeremyMiles conclusion that you have a well-fitting model. Currently, in much of psychometric and social sciences CFA-based research, the Chi-squared is only reported for historical reasons and it is not used as a decisive fit statistic.

I think you will massively benefit from this thread, as it provides you a somewhat succinct summary of each fit statistic, and most importantly for your case, the table of which CFA fit statistic is sensitive to what model condition.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.