0
$\begingroup$

I am running a measurement invariance in lavaan in R. I am comparing a measure of sociocultural appearance pressures across White and Black women. Because this is a 5-point scale, I am treating the items as ordinal. The presented x2 and fit indices were from the robust (WLSMV) estimator.

I have large, but uneven samples. There are 553 White and 273 Black women. I'm evaluating a combo of change in x2 and fit indices. The fit indices are generally adequate for the initial cfa: RMSEA=.085, 95% CI [.081, .090], TLI=.983; SRMR=.054, χ2(199)=1346.800, p<.001.

The configural model is somewhat similar: RMSEA=.088, 95% CI [.083, .092]; TLI=.982; SRMR=.061, χ2(398)=1612.698, p<.001.

The metric model had the following changes: ΔRMSEA=-.009, ΔTLI=.001, and ΔSRMR=.002. Only the SRMR suggested a decrement in model fit, but the change (.002) was below the recommended cut-off of .025 by Chen (2007). The adjusted chi-square difference test was significant, Δχ2(22)=38.62308, p=.016 (I used the scaling correction factors, etc. to calculate the change in the robust x2 estimates).

This is a tricky one for me. I see so much controversy over using the change in chi-square. The fit indices changes were so minimal. My co-author wants us to ignore the chi-square.

I start to question my data when I run the scalar model. In the scalar model, I constrained loadings and thresholds to equality. With this model, I was surprised to see that chi-square goes down, from 1620.556 (420 df) in the metric model to 1462.593 (481 df) in the scalar model. When I run the x2 difference test, I get a p value of 1 because the chi-square went down instead of up. Again, the fit indices changes are minimal, compared to the metric model: ΔRMSEA=-.002, ΔTLI=.003, and ΔSRMR=-.001.

So, I guess my questions are: do you all agree that I should ignore the significant change in x2 from the configural to metric model? And is it OK for the chi-square to decrease from the metric model to the scalar model?

Thank you!

$\endgroup$
0
$\begingroup$

Welcome to CrossValidated!

Your chi-square is very high, and your very good incremental fit indices are also good; whereas chi-square and RMSEA are not good.

This is telling you that you have very high reliabilty - your loadings are high (or I would be surprised if they are not) and this increases the power of the chi-square test to reject models.

That's a very large change in df, and a very large number of df for the model.

How are you comparing chi-squares to get the difference test?

$\endgroup$
  • $\begingroup$ Hi, thanks for responding! I actually used a method that I think you recommended in another post. I used the same syntax and plugged in the robust df, scaling correction factor, and chi-square from the weak and configural models to get the adjusted chi-square difference. Comparing strong and weak models gives me a negative chi-square and a p value of 1, which seems wrong. Additionally, latent factor means significantly vary by group, but if you look at fit indices changes, they seem to improve with the strong model, generally. I'm just unsure how to proceed with interpretation. $\endgroup$ – Blair Burnette Feb 26 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.