Background: I'm running a multi-group model in lavaan. It has two groups: One clinical sample (n = 160) and one healthy control sample (n = 248) that I chose to be comparable regarding age, gender and SES. I'm looking at their scores in several intelligence tests, which form three latent intelligence domains/factors. To check for measurement invariance, I first ran my model without any constraints and got good fit. Then I constrained the factor loadings of all subtests to be equal across the two groups. I was not surprised when the constrained model turned out to have significantly worse fit - it makes sense to me that intelligence subtests in a clinical group could have different loadings on intelligence factors compared to a healthy group.
The thing that puzzled me: I wanted to find out which of my indicator variables' loadings differed so much between groups that I didn't get metric invariance. Some of the loadings were very similar between the groups (differences in standardized factor loadings of around .02 in the cases of lns or dsf, for example), which is why I expected only some of them to make the difference. But even after setting all of them free between the groups (with the group.partial argument in lavaan), except one that differed at .02 between the groups, the constrained model was still significantly different according to a chi-square difference test.
My question: Is the reason for the nonexistent metric invariance really that the small difference in factor loadings (.02) between the two groups is significant? Or am I missing some obvious reasons for the metric measurement variance between my two groups? Reasons I thought of were the different group sizes, or the different variances of the indicator variables between the groups, but my stat knowledge is too tiny to explain why they (or something else) would lead to this. Here's the relevant output for the multigroup model with no constraints:
Group 1: Latent Variables: Estimate Std.Err z-value P(>|z|) Std.lv Std.all PS =~ cd 12.620 0.994 12.693 0.000 12.620 0.890 ss 4.789 0.540 8.865 0.000 4.789 0.634 WM =~ lns 3.686 0.258 14.302 0.000 3.686 0.860 dsf 1.295 0.118 10.932 0.000 1.295 0.662 dsb 1.731 0.169 10.240 0.000 1.731 0.709 Gf =~ mr 4.696 0.208 22.621 0.000 4.696 1.000 Group 2: Latent Variables: Estimate Std.Err z-value P(>|z|) Std.lv Std.all PS =~ cd 11.031 0.918 12.017 0.000 11.031 0.862 ss 6.477 0.487 13.296 0.000 6.477 0.917 WM =~ lns 3.923 0.255 15.370 0.000 3.923 0.844 dsf 1.218 0.171 7.125 0.000 1.218 0.654 dsb 1.472 0.146 10.066 0.000 1.472 0.817 Gf =~ mr 6.018 0.277 21.718 0.000 6.018 1.000
Note: I know my model isn't ideal (only one indicator for Gf, sample sizes are really small for multigroup models, etc...). I'm mainly asking this question to get a better understanding of how measurement invariance testing works.
Note II: I'm aware that Chi square difference tests can be biased in larger samples, but I don't think this is the case here as my n isn't really large (?)