1
$\begingroup$

I have conducted a CFA for a one-factor measurement model and then proceeded to do multigroup CFAs to test for measurement variance for gender and age (binary variable, median split). Testing for measurement invariance proceeds as would be expected for gender, but I am getting some strange results when testing for scalar invariance on age. Specifically, the fit indices all show improved fit from the less restrictive model to the more restrictive model. What might be the issue?

Details

CFA is done with the cfa() function in lavaan version 0.6-13 and the fit objects are compared with the compareFit() function in semTools version 0.5-6.

The lavaan model syntax:

ghmodel <- 'harm =~ Q5_col + Q6_col + Q7_col + Q8_col + Q9_col + Q10_col + Q11_col'

The Qx_col variables have 5 ordinal response alternatives coded with the levels 0, 1, 2, 3, 4 in the data frame "gh".

The general CFA model:

ghfit <- cfa(ghmodel, data = gh, ordered = TRUE)

The binary age variable is coded like this:

gh <- gh %>%
  mutate(ageBinary = case_when(Age < 38 ~ 0,
                               Age >= 38 ~ 1))

Now for the two multigroup models for age and the comparison:

ghfitAge2 <- cfa(ghmodel, ordered = TRUE, data = gh, group = "ageBinary", 
          group.equal = "loadings")
ghfitAge3 <- cfa(ghmodel, ordered = TRUE, data = gh, group = "ageBinary", 
          group.equal = c("loadings", "intercepts"))
comp <- compareFit(ghfitAge2, ghfitAge3)
summary(comp)

At the compareFit() stage I get the following warning message:

Warning message: In (function (object, ..., method = "default", A.method = "delta", : lavaan WARNING: Some restricted models fit better than less restricted models; either these models are not nested, or the less restricted model failed to reach a global optimum. Smallest difference = -25.2639971395378

Now the models are nested as can be seen from the code and the less restricted model did converge.

Here is some of the output which shows reduced chi-square and higher CFI in the more restrictive ghfitAge3:

Model Df Chisq Chisq diff Df diff Pr(>Chisq) cfi cfi diff
ghfitAge2 34 192.61 0.9988652
ghfitAge3 54 167.35 -39.497 20 1 0.9991890 0.0003238

PS: I am aware of the similar post MultiGroup Factor Analysis CFI gets better as model gets more restricted The poster has experienced improved CFI like here but makes no mention of the chi-square. The accepted answer explains that the increase in CFI could be because the chi-square statistic may increase at a slower rate than the degrees of freedom earned by imposing constraints. However, the chi-square is decreasing in my case so I don't see how that can be the answer here.

Might it be due to some downstream effect of using the "ordered = true" specification in the cfa() function? I have attempted to rerun these model with regular ML and then the more restrictive model experiences reduced fit as expected. However, I do not want to go with ML because the fit for the overall model becomes much worse.

$\endgroup$

1 Answer 1

1
$\begingroup$

Read this:

Wu, H., Estabrook, R. (2016). Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes. Psychometrika, 81, 1014–1045. https://doi.org/10.1007/s11336-016-9506-0

You can use this to help with correct specification:

?semTools::measEq.syntax

$\endgroup$
1
  • $\begingroup$ Thank you, this worked. For info to others: It appears the weird results with improved fit are a consequence of how the handling as ordinal variable (ordinal = TRUE) affects model comparisons in multi-group CFA. The reason for this is described in the Wu & Estabrook reference. By using the measEq.syntax function for invariance testing which incorporates Wu & Estabrook's solution you will avoid it. $\endgroup$
    – André
    Commented Apr 6, 2023 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.