I was hoping for some perspectives on the best practices for establishing configural invariance using ordered indicators with WLSMV estimation. I have a 3-factor scale and I am assessing invariance across two groups. I have some question about whether the factor structure is truly invariant across groups. I know that typically when assessing configural invariance, you are just looking at overall model fit, because there is no model to compare it to. However, I have seen some researchers assess model fit in each group as a primary step.

When assessing configural invariance overall, fit indices seem generally good, though RMSEA is slightly high:

CFI: .993 TLI: .986 RMSEA: .083 SRMR: .028

However, I also subset the data and ran the CFA in each group separately. Here are fit statistics:

Group 1 (n=199) CFI: .994 TLI: .989 RMSEA: .066 SRMR: .026

Group 2 (n=368) CFI: .992 TLI: .984 RMSEA: .091 SRMR: .029

I notice that the RMSEA is considerably higher in Group 2, which is also a bigger sample size. Should I be concerned about the difference in RMSEA? When looking at modification indices, it seems like in Group 1, one of the items might load better on a different factor. The MI isn't super high (10ish), but it's almost 0 in the other group.

Is this concerning enough to consider that the factor structure of this measure is not invariant? Should I have even looked at fit within each group? Thanks for any thoughts! It seems like lack of configural invariance is relatively rare, and I am struggling to find many resources on how to assess it beyond just looking at overall model fit.


I think it's a good idea to look at fit within each group. Configural invariance is (relatively) rare, but it can be interesting.

If often occurs because of interpretation differences. For example, Saucier's big 5 mini-markers scale contains the word 'touchy', to American students, this loads on neuroticism, when I gave it to British students, it loads on agreeableness, because British students interpret it to mean 'tactile'. (Similarly, the word 'fussy' to a British person means 'selective', not concerned about detail.

Health outcome scales in the US often include items about affording health care. To people in other countries, this question is confusing. Similarly, outcome scales can contain items about sexual activity, for the very old and the young, these are irrelevant.

Time can also have an effect. The original Eysenck Personality Inventory, from the 1960s, contains the item "Can you usually let yourself go and enjoy yourself a lot at a gay party?” That means something rather different nowadays.

tl;dr: Think about this as a theoretical / conceptual problem, not a statistics problem.

  • 1
    $\begingroup$ So helpful, thank you! This is really helpful as I think theory would suggest differences in factor structure. Configural noninvariance is something I've seen so rarely that I wanted to check myself on the interpretation. Thanks, again! $\endgroup$ – Blair Burnette Jan 16 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.