# What does it mean to correlate residuals in SEM?

I have been reading a paper by Cole, Ciesla, and Steiger, which argues in many cases allowing residuals to correlate is justified. However, I am not entirely sure what it means for residuals to correlate. A good definition and example would be much appreciated.

Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12, 381–398.

It means that the unexplained variance from two variables are correlated. One way of thinking of this is as a partial correlation.

Say we have two regression equations:

$$Y_1i=\beta1_1 \ X_i+\epsilon1_i$$ $$Y_2i=\beta2_1 \ X_i+\epsilon2_i$$

Both equations have an $\epsilon$ term. If you model that as two equations, that's fine. But what if you model it as one equation - do you want to assume that the $\epsilon$ terms are uncorrelated? If you do, then don't correlate them - as in, don't put estimate a correlation in the residual. Usually you don't, so you'd correlate the residuals.

An example: Say you want to look at the effect of age (in adults) on: speed at running 100m, speed at running 5 miles. I'd expect a negative relationship for both of these, but if you modeled them in one equation, you'd expect unexplained variance in 100m running speed to be correlated with 5 mile running speed, controlling for age - so the residuals are correlated.

You can also think of this in terms of latent variables - there are common causes of the residual for both 100m and 5 mile speeds, and hence you can hypothesize the existence of a latent (unmeasured) variable.

• I've sought to extend your marvellously clear example: "Height is an example of a common cause that produces correlated residuals. Age will be a poor predictor of the running speed of extremely short individuals (who will be slow regardless of age), and they'll tend to have large residuals at both running speeds. By contrast, age will be a relatively better predictor of the speed of individuals of average height, and they'll tend to have smaller residuals at both the both running speeds. Hence, height is a latent variable that produces correlated residuals." Is my extension fully accurate? May 22, 2013 at 2:47
• I think that if you're talking about how a predictor improves as another variable changes, you're talking about a moderator effect. May 22, 2013 at 18:36
• I can see I may have made a misstep there. Would it be broadly correct to say that having a high correlation value for the two residuals means that the people with a high residual for one speed (for whatever reason, maybe height or whatever other latent variable) will tend to have a high residual for the other speed, and the people with a low residual for one speed will tend to have a low residual for the other. And before allowing the residuals to correlate we need to ask ourselves whether there's some good theoretical reason why this should be the case. May 23, 2013 at 2:54
• Yes. But residuals have an average of zero, so its clearer to say positive and negative, rather than high or low. May 23, 2013 at 14:03
• Hi, if I find a positive residual covariance in the first lag of a cross-lagged SEM and a negative residual covariance in the second lag, how can I interpret that? Is there a way to see why it is happening descriptively? Jun 28, 2018 at 20:42