# Linear regression coefficient when residuals are regressed against each other

Suppose we have two variables $$X_2$$ and $$X_3$$ and two regressions are run:

$$X_2=a+bX_3+v_2$$

and

$$X_3=c+dX_2+v_3$$

$$v_2$$ and $$v_3$$ are the observed residuals.

Can we say if we regress $$v_3$$ on $$v_2$$, the coefficient of $$v_2$$ will be $$d$$? Is there a way this can be shown?

• What are $v2$ and $v3$ in your models? They look like error terms, which you cannot observe.
– Ale
Commented Sep 26, 2020 at 8:24
• They are the observed residuals. Sorry forgot to mention. I've added it now. Commented Sep 26, 2020 at 9:08
• Have you tried some analytical solution?
– Ale
Commented Sep 26, 2020 at 14:56
– Ale
Commented Sep 26, 2020 at 22:18
• Yes it does! Thanks! Commented Sep 27, 2020 at 7:27

As a hint, consider this simulation in R:

rm(list=ls())
set.seed(42)

n=1000
x3= rnorm(n)
x2 = 1 + 2 * x3 + rnorm(n)

lm(x2 ~ x3)
Coefficients:
(Intercept)    x3
0.9949       2.0098

# let's store the residual v2:
v2 = lm(x2~x3)$res # now let's consider the second model: lm(x3~x2) Coefficients: (Intercept) x2 -0.4044 0.4014 # and store the residual v3: v3 = lm(x3~x2)$res

# let's regress v3 on v2:
lm(v3~v2)
Coefficients:
(Intercept)    v2
-2.255e-17   -4.014e-01


They're not equal but they look related.

Edit:

Plug the equation of $$X_2$$ in the second model and isolate $$v_3$$. You will see that the parameter on $$v_2$$ is $$-d$$.

From the first regression, we have $$X_3 = -\frac{a}{b} + \frac{1}{b} X_2 - \frac{v_2}{b}$$ Comparing this with $$X_3 = c + dX_2 + v_3$$

and assuming the two regressions are performed similarly, we have $$c = -\frac{a}{b}$$ $$d = \frac{1}{b}$$ and $$v_3 = \frac{-1}{b}v2$$

Therefore we have $$v3 = -d v2$$