2
$\begingroup$

I decided to use a questionnaire published by another researcher (paper and supplementary here).

In the article they perform an EFA, find two factors, and report the resulting factor loadings (correlations between the factor and each item). I gave the same questionnaire (in a different context) to a bunch of people and am now wondering if the scale still works the same way as reported in the original paper.

The way to confirm this, from what I understand, is to create a SEM model using the originally reported factor structure, run a CFA, and look at the goodness of fit. What I don't understand is which factors to fix in this model and which I should vary.

My understanding is that I'm asking "Is the originally reported model / factor structure a good fit for my data?" and hence I would fix the loading of each item on each factor to the reported value, and further not add a connection whenever the item doesn't load on a factor according to the original paper. (That would mean I end up fixing all path coefficients except for individual variances??)

However, I don't feel that confident about my knowledge, so I'd like to ask the community of wise statisticians to enlightenment me :)

$\endgroup$
2
$\begingroup$

If you don't have the original data, your approach makes sense. You would need to take the parameters from the original model, then constrain each of the model parameters in the new model. The most strict test would be to constrain all model parameters to what the original paper found. This would be everything: factor loadings, residual variances, factor variances, factor covariances, and residual covariances (potentially factor means as well). You could treat this like a multi-group model (measurement invariance testing). If you acquire bad model fit with the strict model, you could relax some parameters in waves. You could follow the guidelines by this paper by Schoot, Lugtig, and Hox (2012) which outlines the order of parameters to constrain in multi-group models. I would do it backwards in your situation because you are starting from a highly constrained model and working backwards towards a freely estimated model. This approach will allow you to understand which parameters in the models differ.

If you have the original data I see two approaches you could take here:

  1. What you mention. You would fit the original model, and your new data. The corresponding factors (Factor A in the original and Factor A in your data) should have a covariance (correlation) between them fixed to 1. You would do this for each of the factors in the model. I would let the covariances between the factors and all other estimated parameters remain the same as in the paper, for each model. I would not call this SEM as we are not making any predictions to endogenous factors, simply testing the model fit when we imply these factors should be identical (covariance of 1). If model fit is bad, then your factors are different then theirs.

  2. I would prefer a multi-group CFA approach. In this approach you would have more flexibility to constrain individual parameters between the models to see where the models differ. Depending on the software you use, lavaan in R makes this easy. The factor covariances may be the same (as in they pass method 1's fit test), but you may find they differ in their variances, factor loadings, or covariances with other factors within the same model. The multi-group approach will provide a more complete picture of the differences between the models.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ I don't think OP has access to the original data set. They just want to see if the factor structure in their dataset matches that reported in the paper. Fitting a multi-group CFA or setting the correlation between corresponding factors to 1 would not be possible. $\endgroup$ – Noah Jul 18 '19 at 17:45
  • $\begingroup$ Ah, in that case you are correct. I will edit my response to include an alternative $\endgroup$ – Roman_Numerals Jul 18 '19 at 17:47
  • $\begingroup$ @Noah is correct. I don't have access to the original data. $\endgroup$ – FirefoxMetzger Jul 18 '19 at 18:13
  • $\begingroup$ Then the first half of my response applies to you, constrain everything to the original papers values, then work backward freeing parameters as per the Schoot et al paper to find where your model parameters differ. $\endgroup$ – Roman_Numerals Jul 18 '19 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.