I have been trying to understand the use of Exploratory Factor Analysis for the purpose of scale validation.

Say you have developed a scale to measure construct X, which is supposed to be one unified construct, but your EFA tells you that you actually have 4 factors instead of 1 factor.

Wouldn't you always be able to claim that your construct is valid by fitting a second-order CFA with the EFA factors at level 1 and a single factor at level 2?

And in the latter case, your estimate for the level 2-latent variable is going to equal some weighted sum of the original indicators anyway?


Starting with your latter point first, it's not the case that scores for the higher-order factor would be calculated using some sort of crude weighted sum. Unique/error variance in your indicators for the first-order factors is going to get left behind, and only the shared variance in the first-order factors is going to be captured by the higher-order factor, so only in the (unlikely) case that all observed variance in your indicators was shared variance from the higher-order factor would something like a sum-score work. And even then, the construction of sum scores on the basis of a true latent variable requires an awfully strange measurement model (Rose et al., 2019).

The matter of providing strong evidence of a higher-order factor is, itself, also more difficult than many appreciate, as one must appraise its model adequacy against numerous competitors (e.g., bifactor models, correlated lower-order factor models, single factor models, exploratory structural equation models, etc). A few references/examples to consider in this process are Brunner et al. (2012), Chen et al. (2006), Chen et al. (2012), Credé & Harms (2015), Morin et al. (2016; highly recommended), Mulaik & Quartetti (1997), and Wiesner et al. (2013).

Finally, I will note that in my own personal experience, many researchers are (too) comfortable creating "overall" scores (sums or averages) of ostensibly higher-order/general factors on the basis of correlated factor models without ever actually directly appraising the adequacy of higher-order/bifactor models.


Brunner, M., Nagy, G., & Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846.

Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225.

Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., & Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80, 219-251.

Credé, M., & Harms, P. D. (2015). 25 years of higher‐order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational Behavior, 36, 845-872.

Morin, A. J., Arens, A. K., & Marsh, H. W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal, 23, 116-139.

Mulaik, S. A., & Quartetti, D. A. (1997). First order or higher order general factor?. Structural Equation Modeling: A Multidisciplinary Journal, 4, 193-211.

Rose, N., Wagner, W., Mayer, A., & Nagengast, B. (2019). Model-Based Manifest and Latent Composite Scores in Structural Equation Models. Collabra: Psychology, 5.

Wiesner, M., & Schanding, G. T. (2013). Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: Application to the BASC-2 behavioral and emotional screening system teacher form. Journal of School Psychology, 51, 751-763.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.