1
$\begingroup$

For a regression

$$y_t = x_t' \beta + u_t,$$

let $H$ be the hat matrix such that $\hat y = Hy$.

Let $M = I - H$.

I am trying to understand the proof $E[\hat u (b-\beta)'] = 0$.

I was able to understand $E[\hat u (b-\beta)'] = E[Muu'X(X'X)^{-1}]$, but couldnt understand why the following step holds $$E[Muu'X(X'X)^{-1}] = \sigma^2MX(X'X)^{-1} = 0$$ Why is this the case? Could anyone please help?

$\endgroup$
1
$\begingroup$

For the first equality, note that - this is not stated explicitly in your assumptions, but likely mentioned somewhere in your source - $E(uu')=Cov(u)=\sigma^2I$. Also, it appears that $X$ is treated as nonstochastic, so that it may be taken out of the expectation operator. Hence,

$$E[Muu'X(X'X)^{-1}] = ME[uu']X(X'X)^{-1} = M\sigma^2IX(X'X)^{-1}= M\sigma^2X(X'X)^{-1}$$

For the second equality, recall that $H=X(X'X)^{-1}X'$, so that $$MX=(I-H)X=X-X(X'X)^{-1}X'X=X-X=0.$$

From an intuitive perspective, $M$ is the "residual-maker matrix". That is, for a vector $y$, $My$ gives the residuals of regressing $y$ on $X$. Hence, the columns of $MX$ give the residuals of regressing each column in $X$ on $X$. That is, we consider the residuals ("mistakes made") when explaining a column of $X$ by, among amothers, itself. Now, it is intuitive that you do not make a mistake (get zero residuals) when explaining something by itself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.