5
$\begingroup$

I have been reading a paper by Cole, Ciesla, and Steiger, which argues in many cases allowing residuals to correlate is justified. However, I am not entirely sure what it means for residuals to correlate. A good definition and example would be much appreciated.

Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12, 381–398.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
8
$\begingroup$

It means that the unexplained variance from two variables are correlated. One way of thinking of this is as a partial correlation.

Say we have two regression equations:

\begin{equation} Y_1i=\beta1_1 \ X_i+\epsilon1_i \end{equation} \begin{equation} Y_2i=\beta2_1 \ X_i+\epsilon2_i \end{equation}

Both equations have an $\epsilon$ term. If you model that as two equations, that's fine. But what if you model it as one equation - do you want to assume that the $\epsilon$ terms are uncorrelated? If you do, then don't correlate them - as in, don't put estimate a correlation in the residual. Usually you don't, so you'd correlate the residuals.

An example: Say you want to look at the effect of age (in adults) on: speed at running 100m, speed at running 5 miles. I'd expect a negative relationship for both of these, but if you modeled them in one equation, you'd expect unexplained variance in 100m running speed to be correlated with 5 mile running speed, controlling for age - so the residuals are correlated.

You can also think of this in terms of latent variables - there are common causes of the residual for both 100m and 5 mile speeds, and hence you can hypothesize the existence of a latent (unmeasured) variable.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ I've sought to extend your marvellously clear example: "Height is an example of a common cause that produces correlated residuals. Age will be a poor predictor of the running speed of extremely short individuals (who will be slow regardless of age), and they'll tend to have large residuals at both running speeds. By contrast, age will be a relatively better predictor of the speed of individuals of average height, and they'll tend to have smaller residuals at both the both running speeds. Hence, height is a latent variable that produces correlated residuals." Is my extension fully accurate? $\endgroup$
    – user1205901 - Слава Україні
    May 22, 2013 at 2:47
  • $\begingroup$ I think that if you're talking about how a predictor improves as another variable changes, you're talking about a moderator effect. $\endgroup$
    – Jeremy Miles
    May 22, 2013 at 18:36
  • $\begingroup$ I can see I may have made a misstep there. Would it be broadly correct to say that having a high correlation value for the two residuals means that the people with a high residual for one speed (for whatever reason, maybe height or whatever other latent variable) will tend to have a high residual for the other speed, and the people with a low residual for one speed will tend to have a low residual for the other. And before allowing the residuals to correlate we need to ask ourselves whether there's some good theoretical reason why this should be the case. $\endgroup$
    – user1205901 - Слава Україні
    May 23, 2013 at 2:54
  • $\begingroup$ Yes. But residuals have an average of zero, so its clearer to say positive and negative, rather than high or low. $\endgroup$
    – Jeremy Miles
    May 23, 2013 at 14:03
  • $\begingroup$ Hi, if I find a positive residual covariance in the first lag of a cross-lagged SEM and a negative residual covariance in the second lag, how can I interpret that? Is there a way to see why it is happening descriptively? $\endgroup$
    – Blain Waan
    Jun 28, 2018 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.