Why don't I have metric measurement invariance even with very small differences between factor loadings across groups?

Question

Background: I'm running a multi-group model in lavaan. It has two groups: One clinical sample (n = 160) and one healthy control sample (n = 248) that I chose to be comparable regarding age, gender and SES. I'm looking at their scores in several intelligence tests, which form three latent intelligence domains/factors. To check for measurement invariance, I first ran my model without any constraints and got good fit. Then I constrained the factor loadings of all subtests to be equal across the two groups. I was not surprised when the constrained model turned out to have significantly worse fit - it makes sense to me that intelligence subtests in a clinical group could have different loadings on intelligence factors compared to a healthy group.

The thing that puzzled me: I wanted to find out which of my indicator variables' loadings differed so much between groups that I didn't get metric invariance. Some of the loadings were very similar between the groups (differences in standardized factor loadings of around .02 in the cases of lns or dsf, for example), which is why I expected only some of them to make the difference. But even after setting all of them free between the groups (with the group.partial argument in lavaan), except one that differed at .02 between the groups, the constrained model was still significantly different according to a chi-square difference test.

My question: Is the reason for the nonexistent metric invariance really that the small difference in factor loadings (.02) between the two groups is significant? Or am I missing some obvious reasons for the metric measurement variance between my two groups? Reasons I thought of were the different group sizes, or the different variances of the indicator variables between the groups, but my stat knowledge is too tiny to explain why they (or something else) would lead to this. Here's the relevant output for the multigroup model with no constraints:

Group 1:
Latent Variables:
                       Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
      PS =~                                                                 
        cd               12.620    0.994   12.693    0.000   12.620    0.890
        ss                4.789    0.540    8.865    0.000    4.789    0.634
      WM =~                                                                 
        lns               3.686    0.258   14.302    0.000    3.686    0.860
        dsf               1.295    0.118   10.932    0.000    1.295    0.662
        dsb               1.731    0.169   10.240    0.000    1.731    0.709
      Gf =~                                                                 
        mr                4.696    0.208   22.621    0.000    4.696    1.000

Group 2:
Latent Variables:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
  PS =~                                                                 
    cd               11.031    0.918   12.017    0.000   11.031    0.862
    ss                6.477    0.487   13.296    0.000    6.477    0.917
  WM =~                                                                 
    lns               3.923    0.255   15.370    0.000    3.923    0.844
    dsf               1.218    0.171    7.125    0.000    1.218    0.654
    dsb               1.472    0.146   10.066    0.000    1.472    0.817
  Gf =~                                                                 
    mr                6.018    0.277   21.718    0.000    6.018    1.000

---

Note: I know my model isn't ideal (only one indicator for Gf, sample sizes are really small for multigroup models, etc...). I'm mainly asking this question to get a better understanding of how measurement invariance testing works.

Note II: I'm aware that Chi square difference tests can be biased in larger samples, but I don't think this is the case here as my n isn't really large (?)

Answer 1

standardized factor loadings of around .02

Measurement invariance is not a hypothesis about standardized parameters. Standardized loadings can differ even when metric invariance holds, because common- and/or unique-factor variances might differ. If the output you provided is from the configural model, even the unstandardized loadings are not comparable across groups because the common-factor scales are not linked across groups without metric-invariance constraints. A more useful (but still very problematic) method would be to fit a partial-invariance model in which only one indicator's loading varies across groups, so the latent scales are linked and the free loadings are comparable.

Note also that when you constrain loadings to equality, you should only fix the factor variances to 1 in the first group. Otherwise, you are not testing metric invariance alone, but also homogeneity of factor variances. Thus, you cannot compare loadings at all for the single-indicator factor because there is no way to link its scale across groups when the loading is free to vary across groups.