1
$\begingroup$

In a multi group exploratory factor analysis (MG-EFA), we have analysed CFI and RMSEA. A decision has been made to use the criteria from Marsch et al (1) of acceptable fit when CFI > 0.90 and RMSEA < 0.08, and excellent fit when CFI > 0.95 and RMSEA < 0.05. As far as I've understood, this slightly contrasts Hu and Bentler's guidelines of CFI 0.95 and RMSEA 0.06. (2)

Now, my simple questions is regarding to reporting (and thus also about understanding). If a domain for example has excellent fit for CFI and acceptable for RMSEA, do I report is as acceptable, excellent or "excellent fit for CFI, acceptable fit for RMSEA"?

Additionally, should I rather be looking at the confidence intervalls for RMSEA? For example, if RMSEA = 0.058 (CI 0.048-0.069), then I'm guessing this is an acceptable but not excellent fit?

(1) Marsh HW, Hau KT, Wen Z. In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) …. Structural equation modeling. 2004

(2) Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

$\endgroup$

1 Answer 1

3
$\begingroup$

Now, my simple questions is regarding to reporting (and thus also about understanding). If a domain for example has excellent fit for CFI and acceptable for RMSEA, do I report is as acceptable, excellent or "excellent fit for CFI, acceptable fit for RMSEA"?

There are multiple acceptable ways of doing this. You should be fine if you cite the criteria you use. If I were your collaborator, I would suggest something like this - Using the cuff-offs suggested by Marsh, Hau, & Wen (2004), we conclude our MG-EFA model has an acceptable fit since our sample RMSEA is < 0.08 (acceptable fit) and our CFI is > 0.95 (excellent).

Additionally, should I rather be looking at the confidence intervalls for RMSEA? For example, if RMSEA = 0.058 (CI 0.048-0.069), then I'm guessing this is an acceptable but not excellent fit?

Yes, generally speaking, it is typically best to use interval estimates of fit statistics when available (the fact that CIs are easy to calculate for the RMSEA is an advantage it has over other fit indices), so I would suggest using them.

Additionally, the two articles you cite (Marsh, Hau, & Wen 2004; Hu & Bentler, 1999) contain cutoffs that are not valid for your model since you are using an exploratory factor analysis (EFA), NOT a confirmatory factor analysis (CFA). I am not aware of any cutoffs for EFA models (though I am sure some exist), though see Montoya & Edwards (2021) for reasons why fit indices, such as the CFI and RMSEA, are typically not useful for EFA models.

Finally, I feel obligated to mention that the extent to which a particular set of cutoffs are useful depends on how the characteristics of the fitted model align with those used in the simulation studies used to establish the cutoffs. For more information on this, see McNeish & Wolf (2023).

References

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55.

Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural equation modeling, 11(3), 320-341.

McNeish, D., & Wolf, M. G. (2023). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods.

Montoya, A. K., & Edwards, M. C. (2021). The poor fit of model fit for selecting number of factors in exploratory factor analysis for scale evaluation. Educational and psychological measurement, 81(3), 413-440.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.