Let $\gamma$ and $\delta$ denote $K\times 1$ vectors of parameters in models $\hat{Y} = X\gamma+\eta$ and $\hat{\epsilon} = X\delta+\psi$, where $\eta$ and $\psi$ and $n\times 1$ vectors of error terms. Show that the OLS estimators of $\gamma$ and $\delta$ are, \begin{align} \hat{\gamma} &= \hat{\beta}; \\ \hat{\delta} &= 0_{K\times1}; \end{align}

The definitions of $\hat{Y}$ and $\hat{\epsilon}$ are \begin{align} \hat{Y} &=X\hat{\beta} \\ \hat{\epsilon} &= Y-\hat{Y}. \end{align}

I understand that we can show the first one by the following,

$$\hat{\gamma} =(X'X)^{-1}X'\hat{Y} = \hat{\beta}$$

But I'm unable to show the second expression, any guidance or help would be highly appreciated.

  • 1
    $\begingroup$ Please see my answer below. I do not see any connection of the question to unbiasedness (see title), though. $\endgroup$ Dec 9, 2020 at 12:24

1 Answer 1


The residuals $\hat{\epsilon}= Y-\hat{Y}$ can be written as $\hat{\epsilon}=MY$, where $M=I-X(X'X)^{-1}X'$ is the so-called "residual-maker matrix". Hence, using the OLS formula, we have $$ \hat\delta=(X'X)^{-1}X'MY $$ Thanks to symmetry of $M$, we have $X'M=X'M'=(MX)'$. Now, it is well-known that $$ MX=(I-X(X'X)^{-1}X')X=X-X(X'X)^{-1}X'X=X-X=0, $$ so that $\hat\delta=0$, too.

The result makes intuitive sense, too: the residuals $\hat\epsilon$ are what you cannot explain by $X$. If you regress that on $X$ again to get $\hat\delta$, a zero effect makes sense.

  • $\begingroup$ Thank you, this was exactly what I was looking for , I used unbiasedness for lock of better word to explain my problem. My apologies if it was confusing, again thank you for the answer and for taking your time to write it up! $\endgroup$ Dec 9, 2020 at 13:04
  • $\begingroup$ If I may, I'm a bit confused by $\hat\delta=(X'X)^{-1}X'MY$. As the OLS formula is $(X'X)^{-1}X'\hat{Y}$. How do you arrive to the conclusion that $\hat\delta=(X'X)^{-1}X'MY$? $\endgroup$ Dec 9, 2020 at 13:11
  • $\begingroup$ Figured it out, since we have $\hat{\epsilon}=MY$ and not $Y$ in the equality of our model. Dumb question, thanks again! (To much rehearsal and to little attention to learning...) $\endgroup$ Dec 9, 2020 at 13:13
  • $\begingroup$ @MisterButter, very good. I took the liberty to edit the title of the post to maybe better reflect its content, so please have a look if that is fine with you! $\endgroup$ Dec 9, 2020 at 13:39
  • $\begingroup$ Thank you very much! $\endgroup$ Dec 9, 2020 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.