2
$\begingroup$

Suppose I would like to use sumscores after running a confirmatory factor analysis (CFA) with two latent factors. The items for each factor are then summed and in subsequent analyses these sums are used as dependent variables. I am aware of the problems involved when using sumscores in general, but let's assume I would still choose using sumscores instead of more advanced SEM models for my subsequent analyses. However, I'm wondering about the following. The CFA gave a good fit (rmsea, CLT, TLI, normed chisquare) after letting the error terms of two items of one of the factors correlate. Without allowing for this particular correlation, the fit was sub-optimal. What additional problems, if any, would this correlation create when using the sumscores for the subsequent analyses (regression models)? And would e.g. using saved factorscores instead of sumscores (perhaps partially) solve these problems? Thanks for any guidance!

$\endgroup$

1 Answer 1

2
$\begingroup$

What additional problems, if any, would this correlation create when using the sumscores for the subsequent analyses (regression models)?

Correlated errors are evidence of multidimensionality. It is statistically equivalent to replace a residual correlation with an orthogonal factor on which those 2 items load. Even if that other factor is just a source of systematic measurement error, it is compounded in the composite when it is present in more than one item. See the mathematics here:

Little, T. D., Rhemtulla, M., Gibson, K., & Schoemann, A. M. (2013). Why the items versus parcels controversy needn’t be one. Psychological Methods, 18(3), 285–300. https://doi.org/10.1037/a0033266

How that effects your estimated regression coefficients depends on whether/how the non-target factor correlates with the other variables in your regression model. Refer to the causal-inference literature about colliders, confounders, instrumental variables, and such.

would using saved factor scores instead of sum scores (perhaps partially) solve these problems?

The factor scores would exclude variance due to the non-target factor, even if you only estimated the residual covariance. But then your inferences in factor-score regression would need to account for the fact that factor scores are estimates (with their own SEs), not observed data. A plausible-values approach could help (http://www.statmodel.com/download/Plausible.pdf ; for lavaan, see semTools::plausibleValues documentation), or Croon's correction (see also the recently developed SAM technique: https://osf.io/pekbm/

$\endgroup$
4
  • $\begingroup$ Terrence, great answer, thanks a lot!! So, in general this seems an additional argument for using factor scores instead of sumscores, as the influence of the non-target factor is filtered from the factorscores, which are then "cleaner", whereas sumscores are "contaminated". B.t.w. I noticed that the loadings of the two items with correlating errors, got closer to zero after allowing their residuals to correlate in the confirmatory factor analysis. Is this a general trend, you think, or coincidental? I expected this to happen but I'm not sure. Thanks again. $\endgroup$
    – BenP
    Feb 4, 2022 at 14:39
  • $\begingroup$ Yes, that is expected because of propagation of errors. Without the residual covariance, those 2 items can't correlate enough, so their factor loadings are increased to make up for that (which should then decrease other items' loadings). $\endgroup$
    – Terrence
    Feb 11, 2022 at 12:43
  • $\begingroup$ Thanks again Terence for your answer. But as is often the case, each good answer raises a new question. You previously said that allowing for correlated errors is equal to modeling a second factor, orthogonal to the primary factor, on which the two items load. That makes sense. But what would happen if this second factor is correlated with the primary factor? Would that mean that allowing correlated errors is not a solution for that situation, and hence, that allowing the two item residuals to correlate would still lead to wrong or biased factor loading on the primary factor? $\endgroup$
    – BenP
    Feb 11, 2022 at 21:32
  • $\begingroup$ Problem remains, but diminishes as the factors become more redundant (positively correlated). But I'm not sure your model would be identified if you tried to estimate the factor correlation, and it would be less parsimonious than the correlated residuals (which is equivalent only to an additional orthogonal factor). $\endgroup$
    – Terrence
    Feb 13, 2022 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.