Multisample SEM in R: Differences in significance level using UVI vs ULI I'm a graduate student in psychology trying to examine the factor invariance of some psychological constructs worry (using a two-factor model) using Structural Equation Modelling (SEM) in R, using the lavaan package. I am comparing two groups (male & female). 
According to Beaujean (2014), there are different types of measurement invariance (1. Configural invariance, 2. Weak invariance, 3. Strong invariance etc). Different genders survived Configural invariance by demonstrating a good fit in terms of CFI, SRMR, RMSEA etc. However, upon testing weak invariance (testing whether the factor loadings are the same across groups), the model doesn't show a good fit.
To see where the discrepancies lie, I then compared the estimates of factor loadings in males and females using Unit Variance Identification.
CFA.model.UVI <- '
# measurement equations
Factor1 =~ c(m1,f1)*item_1 + c(m2,f2)*item_2 + c(m3,f3)*item_3
Factor2 =~ c(m4,f4)*item_4 + c(m5,f5)*item_5 + c(m6,f6)*item_6

# compare estimates
diff1 := m1 - f1
diff2 := m2 - f2
diff3 := m3 - f3
diff4 := m4 - f4
diff5 := m5 - f5
diff6 := m6 - f6
'

I examined the significance level of diff1 to diff6 and found that most of them are NOT significantly different. HOWEVER, I then fit a Unit Loading Identification (ULI) model.
CFA.model.ULI <- '
# measurement equations
Factor1 =~ 1*item_1 + c(m2,f2)*item_2 + c(m3,f3)*item_3
Factor2 =~ 1*item_4 + c(m5,f5)*item_5 + c(m6,f6)*item_6

# compare estimates
diff2 := m2 - f2
diff3 := m3 - f3
diff5 := m5 - f5
diff6 := m6 - f6
'

Surprisingly, the model shows that diff2 and diff3 are significantly different but diff5 and diff6 are not. In other words, UVI and ULI give different estimates and different significance levels.
I have two questions:


*

*Is it legitimate for me to compare estimates of two groups?

*If it is legitimate, which model (UVI or ULI) should be more reliable?


Thanks so much.
 A: Brief response to your two queries: yes, and UVI. But, as brief responses...there are many scenarios where these answers are actually incorrect.
I'll elaborate more, but in the reverse order you asked the questions.
The issue you raise is less about reliability with the models, and more about the estimation process and structure (artificially) imposed on the latent variables. The latent variables can be anchored to one item (meaning the latent variable shares the same scale) or standardized like a $z$-score. And, it is possible for the resulting models to produce different cross-group results. So, in this case, you should focus on what you are asking: ¿are the factor loadings the same for the groups? Because of this, you can answer this better by fixing the variance and freely estimating the loadings. In the ULI model, you essentially go in with the assumption that item 1 has the same factor loading.  And if it doesn't, this means the latent variables will be different for each group.
So, if you run the UVI, you can indeed compare estimates for each group. However, I would suggest a more conventional test.  If you decide that an item may have different factor loadings, then do a model comparison. Run the UVI model with all the loadings constrained, and then run another model with only one of the pairs freely estimated for each group. E.g., for all, use c(mf#,mf#); for the second model, go back to c(m#,f#). You can then do a chi-square test to see if the models are statistically significantly different.  If so, they should be estimated separately.  If not, they can be equated and you can move to the next model invariance test.
Hope this helps.
