2
$\begingroup$

I would like to know your opinion about the following issue.

I am estimating a multi-group SEM model on two groups (urban vs. extra-urban). The theory behind states that the factor loadings could differ between the two groups; indeed, the model with configural invariance (no equality constraints on factor loadings) has a better fit than the model with same factor loadings in the two groups. This should point in the direction I expected, right?

Also, I would like to speculate a bit on the differences in factor loadings between the two groups: how can I test the statistical significance of those differences? What kind of test should I run?

Thanks a lot in advance, best, G.P.

$\endgroup$

1 Answer 1

2
$\begingroup$

Welcome gipicci!

You have already fit your two competing models--in your case, the configural and loading invariant models. These models are "nested", in that they contain the same observed indicators, and the "nested" model (loading invariant) is a more parsimonious/constrained version of the more complicated "full" or "parent" model. Subsequently, the most common way of testing for differences in loadings would be to perform a $\chi^2$ difference test, where you compute and test a new $\chi^2$ statistic as: $\chi^2$(loading invariant) - $\chi^2$ (configural), with degrees of freedom $df$(loading) - $df$(configural).

However, you may (or may not) already be familiar with some of the concerns/objections to relying heavily on the $\chi^2$ statistic in model evaluation, and so there are other ways you could approach evaluating the comparability of loadings, such as by monitoring the increase in $RMSEA$ or the decrease in $CFI$ (see Chen, 2007; Cheung & Rensvold, 2002).

All of these approaches, so far, could be considered as something akin to an ANOVA for loadings--you are performing an omnibus test of the equivalence of all loadings simultaneously. If it was of interest to you to see which loading(s) were driving the noninvariance, instead of fixing all of them to equivalence, you could fix one particular pair to equivalence and compare against your configural invariance model, and then rinse and repeat for each pair of loadings you could test. Additionally, you could consider calculating an effect size measure of noninvariance, in order to illustrate the magnitude of noninvariance in a given indicator between groups (see Nye & Drasgow, 2011).

References

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255.

Nye, C. D., & Drasgow, F. (2011). Effect size indices for analyses of measurement equivalence: Understanding the practical importance of differences between groups. Journal of Applied Psychology, 96(5), 966.

$\endgroup$
2
  • $\begingroup$ Dear jsakaluk, thanks a lot for your explanation. I am quite new to this technique, so sorry if I ask you something silly to you, but I would like to know 1) How can I perform what you call the chi square difference test? 2) I had a look at your references, and they are very interesting, but I was not able to find something related to the issue of my question: if I find that the two group have a configural model, but not equality of factor loadings, how can I test if the differences of factor loadings in the two groups are statistically significant? Thanks a lot $\endgroup$
    – gipicci
    Commented Jun 26, 2019 at 9:41
  • $\begingroup$ For 1) Lots of programs automate this nested model comparison process, but if you need to do it on your own, you can just use excel: calculate the difference of chi-squares, and the difference of dfs, and use excel's chi-square function submitting the chi-square difference as the new chi-square, and the df difference as the new df. This is the omnibus "significance" test that most would report when addressing 2) $\endgroup$
    – jsakaluk
    Commented Jun 26, 2019 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.