# OLS coefficients of regressions of fitted values and residuals on the original regressors

Let $$\gamma$$ and $$\delta$$ denote $$K\times 1$$ vectors of parameters in models $$\hat{Y} = X\gamma+\eta$$ and $$\hat{\epsilon} = X\delta+\psi$$, where $$\eta$$ and $$\psi$$ and $$n\times 1$$ vectors of error terms. Show that the OLS estimators of $$\gamma$$ and $$\delta$$ are, \begin{align} \hat{\gamma} &= \hat{\beta}; \\ \hat{\delta} &= 0_{K\times1}; \end{align}

The definitions of $$\hat{Y}$$ and $$\hat{\epsilon}$$ are \begin{align} \hat{Y} &=X\hat{\beta} \\ \hat{\epsilon} &= Y-\hat{Y}. \end{align}

I understand that we can show the first one by the following,

$$\hat{\gamma} =(X'X)^{-1}X'\hat{Y} = \hat{\beta}$$

But I'm unable to show the second expression, any guidance or help would be highly appreciated.

• Please see my answer below. I do not see any connection of the question to unbiasedness (see title), though. Dec 9 '20 at 12:24

The residuals $$\hat{\epsilon}= Y-\hat{Y}$$ can be written as $$\hat{\epsilon}=MY$$, where $$M=I-X(X'X)^{-1}X'$$ is the so-called "residual-maker matrix". Hence, using the OLS formula, we have $$\hat\delta=(X'X)^{-1}X'MY$$ Thanks to symmetry of $$M$$, we have $$X'M=X'M'=(MX)'$$. Now, it is well-known that $$MX=(I-X(X'X)^{-1}X')X=X-X(X'X)^{-1}X'X=X-X=0,$$ so that $$\hat\delta=0$$, too.
The result makes intuitive sense, too: the residuals $$\hat\epsilon$$ are what you cannot explain by $$X$$. If you regress that on $$X$$ again to get $$\hat\delta$$, a zero effect makes sense.
• If I may, I'm a bit confused by $\hat\delta=(X'X)^{-1}X'MY$. As the OLS formula is $(X'X)^{-1}X'\hat{Y}$. How do you arrive to the conclusion that $\hat\delta=(X'X)^{-1}X'MY$? Dec 9 '20 at 13:11
• Figured it out, since we have $\hat{\epsilon}=MY$ and not $Y$ in the equality of our model. Dumb question, thanks again! (To much rehearsal and to little attention to learning...) Dec 9 '20 at 13:13