Do not know how to regress, I have trouble understanding Frisch-Waugh-Lovell

Question

Hello everyone I am an MSc student in economics and one of my modules is Econometrics I which seems to be proof based or what I think could be considered proof-based. Before asking my question here I asked my professor but he's in vacation and don't know when he's going to answer

Now, some of you may laugh at me but I having a hard time understanding a particular answer. So I have this:

and this is the answer:

I don't understand what exactly are the steps he's taking, what does he mean to Regress X1 on X2? and how does he get that gamma hat? Sorry if these are silly questions but most of his answers are usually step-by-step and have no trouble understanding.

Also another questions is related to estimation of variance which is:

An unbiased estimator $\sigma^2 is: $s$^2$ = $\frac{SSR}{(n-k)}$ so the variance of $\beta$ hat = $s^2$(X'X$^-$$^1$)

Where does that s^2 come from? what is it's proof? and what exactly does it mean? What does s equal to? I apologize i'm asking this but I got an exam next year in early January and want to be prepared.

If someone can suggest a proof heavy book in econometrics that shows steps and whatnot it would be great since I mostly use the lecture slides which are really good.

I apologize for the long post.

And before anyone asks, I have a bad habit of not reading during the term/semester and end up asking questions barely at the end.

Answer 1

I know the feeling you have. Just a couple of weeks ago I had problems to grasp the idea of FWL. What helped me a lot was using $Z$ (instead of $X_2$) and $X$ (instead of $X_1$), so I'll try to explain it in that fashion.

Starting with step 1, regressing $X$ on $Z$ means actually regressing each column of $X$ on the whole matrix $Z$, i.e. one considers $$ X = Z\Gamma + V. $$ Note that $\Gamma$ and $V$ are matrices (hence the capital letters) since the dependent variable is a matrix; cp. https://en.wikipedia.org/wiki/General_matrix_notation_of_a_VAR(p) (eventhough this is related to time series analysis but the notation is the same). The OLSE of $\Gamma$, $\hat\Gamma$, is given by $$\hat\Gamma = (Z'Z)^{-1}Z'X.$$ Why? Well, you can either check the source given in the link above (Lütkepohl) or try it on your own by treating $X$, $\Gamma$ and $V$ as vectors (eventhough they are matrices!) and perform OLS. You will get $\hat\Gamma$ as stated in the latter equation. Now calculate the residuals (which works again just as in the case with a single vector as dependent variable): $$\hat V = X - Z\hat\Gamma \\\quad\qquad\qquad= X - Z(Z'Z)^{-1}Z'X\\\quad\quad=(I-P_Z)X\\=M_ZX,$$ where $P_Z = Z(Z'Z)^{-1}Z'$ and $M_Z = I-P_Z$.

The second step tells you to regress the actual dependent variable, $y$, on the residuals of the model calculated in step 1, i.e. $$y = \hat V\beta + u\\\qquad=M_ZX\beta + u.$$ Now you can already tell what the OLSE of this model is. It is $$\hat\beta = (\hat V'\hat V)^{-1}\hat V'y\\\qquad\qquad=(X'M_Z'M_ZX)^{-1}X'M_Z'y\\\quad\qquad=(X'M_ZX)^{-1}X'M_Zy$$ since $M_Z=M_Z'$ and $M_ZM_Z=M_Z$. And finished :-)

Regarding your second question: You may know that the estimator $\hat\sigma^2=\hat u'\hat u = SSR$ is biased since $E(\hat\sigma^2) = \sigma^2(n-k)$. If you replace $\hat\sigma^2$ by $s^2 = SSR/(n-k)$ you will get $E(s^2) = \frac{\sigma^2}{n-k}(n-k) = \sigma^2$ (unbiasedness).