# Where does this instrumental variables transformation come from?

I am reading on instrumental variables estimation in linear regression, and specifically, two stage least squares estimation.

Assume we have a model $$y=X\beta+\epsilon$$ Where $X$ is correlated with $\epsilon$ but we have instrumental variables $Z$ that are not correlated with epsilon, but with $X$.

The book by Greene (Econometric Analysis), states that, when using an instrumental variables estimate, it is best to not use the simplest form of an instrumental estimate, namely $\hat \beta_{IV}=(Z^TX)^{-1}Z^Ty$, and that in any case this isn't possible if $Z$ has higher rank than $X$.

Instead we should use more of the information that is in $Z$, by converting it into $\hat X$ as follows:

My question is: Why is this called the "projection of the columns of $X$ in the column space of $Z$"? How do we come up with this? (i.e. how would one come up with this without simply assuming it before hand, solely based on the goal of finding the best instrumental variable estimator)?

The book by Green doesn't justify this at all.

$\hat X$ is just the predicted value for X from a linear regression of $X$ on $Z$. There's an geometric interpretation of least squares, with $\hat X$ as the orthogonal projection of $X$ onto the subspace the column vectors of $Z$.

It's usually explained in terms of $Y$ and $X$, but the idea is very similar. Here's a illustration that conveys the idea of why you can think of this as a projection:

How do we come up with this?

We come up with this if we suspect (following the results of a Durbin–Wu–Hausman test) that $\boldsymbol{X}$ is correlated with $\boldsymbol{\epsilon}$, i.e. we suspect that $\boldsymbol{X}$ is actually not exogenous because something that could explain any or all of its columns (and that we omitted to include in the model) is actually not captured and thus goes in $\boldsymbol{\epsilon}$. Indeed, the only variable that is "allowed" to be endogenous is $\boldsymbol{y}$. In such a case, on the basis of our technical knowledge of the phenomenon under study, one would

(i) find an instrumental (hyper-) variable $\boldsymbol{Z}$,

(ii) use it to estimate the projection of $\boldsymbol{X}$, namely $\widehat{\boldsymbol{X}}$

(iii) estimate the relation $\boldsymbol{y} = \widehat{\boldsymbol{X}}\boldsymbol{\beta} + \boldsymbol{\epsilon}$ instead of $\boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$, in hope that $\widehat{\boldsymbol{X}}$ is henceforth exogenous.

This is a better approach according to Greene (Econometric Analysis), since $\boldsymbol{X}$ and $\boldsymbol{Z}$ do not necessarily have the same number of columns (same rank), by opposition to what requires the computation of $\boldsymbol{\beta}_{IV}$.

Also, note that the estimation can equivalently be performed in one shot, playing directly with

$\boldsymbol{y} = \boldsymbol{Z}(\boldsymbol{Z}'\boldsymbol{Z})^{-1}\boldsymbol{Z}'\boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$

• Any question @user56834 ? Oct 17, 2019 at 11:42