I have a model that has data from two different groups. As per the suggestion from Beaujean (2014) I run measurementInvariance and find that even the strong fit has good fit indices.

chisq  df   pvalue cfi  rmsea   srmr    
47.992 57   0.797  1.0  0.0     0.047  

This suggests that I could merge the data and run a joint model, right? When I run the model jointly (constraining loading and intercepts), one particular loading is significant (p < 0.05). However when I run the two groups separately (unconstrained loading and intercept), that loading becomes significant in one group (p <0.1) and insignificant in another (p>0.1). I understand that the significant / insignificant issue is because of the arbitrary significance level (p = 0.1). But, assuming that I can't fight the arbitrariness, how should I look at and report this issue? Should I run the constrained model only and ignore the difference between the two groups? Or should I run the unconstrained model and claim that for this loading, the two groups are different?


What to report? If you are trying to contribute to knowledge, the answer is, "Everything." Your result is not surprising. The unconstrained model involves estimating many more parameters with the same sample size, and there is more opportunity for dependence across different parameter estimates. Put those together, and one would expect the unconstrained model to report larger standard errors and also larger p-values.

So report the unconstrained results and then the constrained result. My interpretation is that you don't have enough statistical power to detect the difference in loading estimates across groups. You don't mention sample size, but given your low chi-square, I would guess that sample size is not large. Still, your parameter estimates may be interesting to other researchers in your field.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.