OLS coefficients of regressions of fitted values and residuals on the original regressors

Question

Let $\gamma$ and $\delta$ denote $K\times 1$ vectors of parameters in models $\hat{Y} = X\gamma+\eta$ and $\hat{\epsilon} = X\delta+\psi$, where $\eta$ and $\psi$ and $n\times 1$ vectors of error terms. Show that the OLS estimators of $\gamma$ and $\delta$ are, \begin{align} \hat{\gamma} &= \hat{\beta}; \\ \hat{\delta} &= 0_{K\times1}; \end{align}

The definitions of $\hat{Y}$ and $\hat{\epsilon}$ are \begin{align} \hat{Y} &=X\hat{\beta} \\ \hat{\epsilon} &= Y-\hat{Y}. \end{align}

I understand that we can show the first one by the following,

$$\hat{\gamma} =(X'X)^{-1}X'\hat{Y} = \hat{\beta}$$

But I'm unable to show the second expression, any guidance or help would be highly appreciated.

Answer 1

The residuals $\hat{\epsilon}= Y-\hat{Y}$ can be written as $\hat{\epsilon}=MY$, where $M=I-X(X'X)^{-1}X'$ is the so-called "residual-maker matrix". Hence, using the OLS formula, we have $$ \hat\delta=(X'X)^{-1}X'MY $$ Thanks to symmetry of $M$, we have $X'M=X'M'=(MX)'$. Now, it is well-known that $$ MX=(I-X(X'X)^{-1}X')X=X-X(X'X)^{-1}X'X=X-X=0, $$ so that $\hat\delta=0$, too.

The result makes intuitive sense, too: the residuals $\hat\epsilon$ are what you cannot explain by $X$. If you regress that on $X$ again to get $\hat\delta$, a zero effect makes sense.