Whether to use EFA or CFA to predict latent variables scores?

Question

I have a dataframe of individual observations, that I partitioned to create a training (0.7 prop) and a test set (0.3 prop).

I started by running an exploratory factor analysis (EFA) on the training set (with the psych::fa function). Then, based on results, I fitted a CFA on the test set (with lavaan), associating each observed variable with one latent factor based on its maximum loading.

I would like, then, to use these latent variables as new variables for other analyses (using factor analysis as a feature reduction method). I know it's best to do structural equation modelling, as those latent variables include error, but that's not the option I want to pursue in respect to the models I want to fit.

Anyway, should I use the results of the EFA or the CFA to predict the new factors on my initial dataframe?

Answer 1

As Jeremy pointed out, EFA, CFA, and IRT models scores would usually be in close agreement. This is especially true in the case of unidimensional scales, or 2nd-order factor models (since this will take you back to almost the same configuration when working on the higher-order factor). Moreover, PCA, which does not take into account measurement error but is often used to select the number of relevant factors, will also be highly correlated to those factor scores, like is the case for raw summated scale score provided the scale is truly unidimensional --- after all, a simple or weighted sum of all item scores is all what is desired to summarize a latent trait. In the case of multi-dimensional scales, you can consider each scale separately, if this makes sense.

For an illustration, here is one of the three subscales of the Holzinger & Swineford (1939) study, available in lavaan. I choose a simple correlated factor model, although several other CFA models could be built (and would be equally valid). I used the principal axis factoring for extracting factors in the case of (oblique) EFA. Both the EFA and CFA models were estimated on all items (3 subscales). For PCA, I restrict the computation on the single "visual" subscale (to avoid rotation after PCA).

As can be seen in the picture below (EFA factor scores are on the horizontal axis and PCA or CFA scores on the vertical axis), the correlation is above 0.95 in both cases.

Of course, there are many ways to construct factor scores in the EFA framework, see, e.g., Understanding and Using Factor Scores: Considerations for the Applied Researcher, by DiStefano et al. I'm almost sure I came across papers dealing with the correlation between EFA and CFA scores but I can't get my hands on it anymore.

It's not so "weird to try both and select the one that works the best" --- what is really problematic is to force a factor structure without testing its relevance on independant samples, this is just capitalizing on chance, IMO: I would simply suggest to use CFA factor scores if the factor structure is already defined, or use EFA scores if the interest is simply in feature reduction (as you would use PCA scores in PCR in a regression context). Differences between EFA and CFA are often overstated as both methods are useful, even in an exploratory approach (CFA has model fit indices, which may be helpful, or not).

Answer 2

It looks as though you are interested in extracting (i.e., observing) latent variable scores for prediction purposes (i.e., not necessarily to make inferences). Given this, I would not rule out PCA either (while dually noting its similarity to EFA; see link below for more details). Also, since your goal prediction, I would not suggest using CFA, as the most interpretable model may not be the best for your purposes.

Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?