Justify the deletion of a factor in EFA

Question

I have done an EFA on 180 observations. In order to explain my latent variables I have considered 24 variables and I get 7 factors. Unfortunately 2 factors among the 7 have a cronbach's alpha around 0.5. So I thought about deleting these factors and then developing a Confirmatory factor analysis CFA. From the CFA I get very good results, CFI and TLI above 0.9 and RMSEA lower than 0.8.

Is it justified the deletion of the two factors? Can I justify it using the Cronbach's alpha?

Thanks in advance!

Answer 1

Exploratory factor analysis (EFA) is chock-full of decision points. I have posted a guide to these at http://yellowbrickstats.com/documents/Factor%20Analysis%20Decision%20Guide.pdf :

A. Decide how to treat missing data
B. Assess data suitability for factor analysis
C. Choose an extraction method
D. Pick a rotation method
E. Decide how many factors to extract
F. Decide which items to keep in the analysis
G. Choose a way of displaying results

Your wording suggests that you have allowed software defaults to make a lot of those decisions for you. I recommend that you document here whatever you can about your code and the decisions you deliberately made (the “specs” you chose).

To conduct EFA on 24 input variables using only 180 observations means that you have sparse data with which to estimate many relationships. I’m not saying this approach is out of the question, but it should be helpful to reduce the number of inputs, ideally on theoretical grounds. Variables that on their face seem unrelated to any other group of variables, or that seem almost completely redundant with other individual ones, would be best dropped. With N=180, your solution(s) would inspire more confidence if the inputs were reduced to perhaps 15. But then, can you describe the audience for this analysis, and what they prioritize?

For choosing the best number of factors to extract, currently the favored methods don’t include the Kaiser-Guttman rule. It is wickedly parodied in Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's electric factor analysis machine. Understanding Statistics, 2, 13-32, http://www.quantpsy.org/pubs/preacher_maccallum_2003.pdf. It's further discredited in Braeken, J., & van Assen, M. A. L. M. (2017). An empirical Kaiser criterion. Psychological Methods, 22(3), 450-466. http://dx.doi.org/10.1037/met0000074. Nor is the scree test among those currently favored. Instead, parallel analysis, the minimum average partial correlation test, and the empirical Kaiser criterion are preferred.

I recommend looking into more of the above-listed aspects of the EFA, and thinking through your results after more iterations, before you proceed to CFA, if you do at all. And then if you do, that you cross-validate to assess the robustness of your CFA solution. Not with simple split-half cross-validation, since an N of 90 will be almost definitely too small for the task at hand.

Your results for CFI and TLI do not sound necessarily “very good” to me. You may want to look further into benchmarks for such indicators.

Finally, I would not say definitely Yes or No on the use of Cronbach's alpha to aid in factor-retention decisions. There are too many other aspects of the process that I would recommend examining further.

Answer 2

In general, the deletion of factors in EFA is justified by low eigenvalues and not alpha values. Better, we tend to retain factors with high enough eigenvalues, and then we check for reliability. In a theoretical sense, it's not very important to check reliability before selecting your factors. Why? Because an EFA tries to explain item variability based on bigger attributes (i.e. factors). When it does that, some factors explain a large part of item variability and others a little. Commonly, factors that don't make theoretical sense are created when you have a large pool of items - which would explain your low alpha values on those 2 factors. I'll briefly explain procedures to retain factors, how this can be done with R, and what I would do if I was in your situation.

Procedures for retaining factors

Brown (2015) details some procedures for retaining factors and provide good references to this problem. According to Brown "three commonly used factor selection procedures are based on eigenvalues. They are (1) the Kaiser–Guttman rule, (2) the scree test, and (3) parallel analysis" (p. 23).

1. Kaiser-Guttman rule: factors with eigenvalues below 1 are deleted. Why? You always have p eigenvalues, being p the number of items or parameters in the model. If one factor has eigenvalue above 1, this factor explains at least one parameter. If one eigenvalue is equal to 1, this factor explains the variation equal to one item. And if an eigenvalue is below 1, this factor can't even explain the variation of one item. Despite its simplicity, this rule is very strict. Let's say you have Factor A with eigenvalue = 1.01 and Factor B with eigenvalue = .99. Following this rule, Factor A would be retained and Factor B would be excluded, even though both factors have similar eigenvalues. Also, sometimes you can have a ton of factors with eigenvalue above 1 that just don't make sense.

2. The scree test: you visually look at the eigenvalues for each factor and notice when there is an abrupt change. You then retain the factors with eigenvalues before this great change happens. Based on this procedure, in the figure below (Brown 2015, p. 24) either a four-factor solution or a five-factor solution can be retained. This goes to show that this approach is heavenly dependent on visual interpretation.

3. Parallel analysis: this procedure creates n random data with the same number of p. Then, the observed sample and the random data eigenvalues are plotted against each other. Then,

...factor selection is guided by the number of real eigenvalues greater than the eigenvalues generated from the random data; that is, if the “real” factor explains less variance than the corresponding factor obtained from random numbers, it should not be included in the factor analysis. (Brown, 2015, p. 24)

The figure below shows an example of parallel analysis. Based on that, since the random dataset obtained five factors with a better eigenvalue than our research data, we would retain 4 factors.

How to do this in R?

1. You can do the Kaiser-Guttman procedure just by checking your eigenvalues.

eigen_matrix <- eigen(correlation_matrix)

eigen_matrix$values

If you're doing this on another software, just check how many factors have eigenvalues above 1.

2. For scree plot in R, you use the same eigenvalues checked before and run:

plot(eigen_matrix$values, type = 'b', main = 'Scree plot')

3. To perform parallel analysis, you could run:

fa.parallel(correlation_matrix, n.obs = 180, fm = 'ml')

Be careful with the factoring method (fm) since this could greatly influence your results based on the type of your input data and its characteristics (normality, skewness, and so on).

What I'd do if I were you

An EFA is a procedure used in reducing a great number of items into smaller factors that make sense. If you've got a good theory on why you're trying to group these items, you could also probably infer if the factors you're working on make sense. I'd perform a parallel analysis and check if this procedure retains sound factors. When you're doing an EFA with various items, it's very common that the results will spit out factors that aren't theoretically coherent. Apples may join with oranges in such cases - and if you retain factors like these, it becomes very easy to question the plausibility of your work. So, in a nutshell: use sound procedures to inform your decisions; but, in the end, above all, theory.

References

Auerswald, M., & Moshagen, M. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468–491. https://doi.org/10.1037/met0000200

Brown, T. A. (2015). Confirmatory Factor Analysis for applied research (2nd ed.). The Guilford Press.

Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in Exploratory Factor Analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191-205. https://doi.org/10.1177/1094428104263675