Do I estimate factor loadings in a confirmatory factor anlysis (CFA) aimed at verifying an exploratory factor analysis (EFA)?

Question

I decided to use a questionnaire published by another researcher (paper and supplementary here).

In the article they perform an EFA, find two factors, and report the resulting factor loadings (correlations between the factor and each item). I gave the same questionnaire (in a different context) to a bunch of people and am now wondering if the scale still works the same way as reported in the original paper.

The way to confirm this, from what I understand, is to create a SEM model using the originally reported factor structure, run a CFA, and look at the goodness of fit. What I don't understand is which factors to fix in this model and which I should vary.

My understanding is that I'm asking "Is the originally reported model / factor structure a good fit for my data?" and hence I would fix the loading of each item on each factor to the reported value, and further not add a connection whenever the item doesn't load on a factor according to the original paper. (That would mean I end up fixing all path coefficients except for individual variances??)

However, I don't feel that confident about my knowledge, so I'd like to ask the community of wise statisticians to enlightenment me :)

Answer 1

If you don't have the original data, your approach makes sense. You would need to take the parameters from the original model, then constrain each of the model parameters in the new model. The most strict test would be to constrain all model parameters to what the original paper found. This would be everything: factor loadings, residual variances, factor variances, factor covariances, and residual covariances (potentially factor means as well). You could treat this like a multi-group model (measurement invariance testing). If you acquire bad model fit with the strict model, you could relax some parameters in waves. You could follow the guidelines by this paper by Schoot, Lugtig, and Hox (2012) which outlines the order of parameters to constrain in multi-group models. I would do it backwards in your situation because you are starting from a highly constrained model and working backwards towards a freely estimated model. This approach will allow you to understand which parameters in the models differ.

If you have the original data I see two approaches you could take here:

1. What you mention. You would fit the original model, and your new data. The corresponding factors (Factor A in the original and Factor A in your data) should have a covariance (correlation) between them fixed to 1. You would do this for each of the factors in the model. I would let the covariances between the factors and all other estimated parameters remain the same as in the paper, for each model. I would not call this SEM as we are not making any predictions to endogenous factors, simply testing the model fit when we imply these factors should be identical (covariance of 1). If model fit is bad, then your factors are different then theirs.

2. I would prefer a multi-group CFA approach. In this approach you would have more flexibility to constrain individual parameters between the models to see where the models differ. Depending on the software you use, lavaan in R makes this easy. The factor covariances may be the same (as in they pass method 1's fit test), but you may find they differ in their variances, factor loadings, or covariances with other factors within the same model. The multi-group approach will provide a more complete picture of the differences between the models.