Instrumental variables equivalent representation

Question

Let us work with the following structural model: $$y=\mathbf{x_{1i}β}+x_{2i}β_2+\varepsilon_i$$

where $x_{2i}$ is our single endogenous regressor. It turns out that given my instruments and my first stage regression, I can obtain $β_2$ in two different ways. I can use the two stage least squares regression $$y=\mathbf{x_{1i}β}+\hat{x}_{2i}β_2+\varepsilon_i \qquad (*)$$ where $\hat{x}_{2i}$ is our instrument predicted with the first stage regression.

And we can obtain the same estimate for $β_2$ with the following regression $$y=\mathbf{x_{1i}\beta}+x_{2i}β_2+\delta\hat{v}_i+\varepsilon_i \qquad (**)$$

where $\hat{v}_i$ is the estimated residual of the first stage regression, namely the portion of the original endogenous regressor that is correlated with the original $\varepsilon_i$.

Now I would like to prove algebraically that we can estimate $β_2$ using these two different approaches. I understand why the estimates must be the same. After all, in $(*)$ we are estimating the partial effect of $x_{2i}$ on $y$ controlling for the portion of $x_{2i}$ that was correlated with the error in the structural model above. Is this interpretation correct? Furthermore I tried to explicit this algebraically. I know I should be able to rewrite $(*)$ as $(**)$ or viceversa. Unfortunately I am not convinced of my result. I started from $(*)$, used the fact that $\hat{v_i}=x_{2i}-\hat{x}_{2i}$ and got the following result $$y=\mathbf{x_{1i}β}+x_{2i}β_2-β_2\hat{v_i}+\varepsilon_i$$

Now, this result is similar to $(**)$ but I cannot understand it. In particular, how should I decompose $x_{2i}$? What am I missing? Could you give me a hint?

Thank you for your help.

Answer 1

When I am trying to figure out why a result is true, I usually look at it in the simplest possible case. So, let's try a bivariate system. Here is the structural system: \begin{align} y_i &= \beta x_i + \epsilon_i\\ x_i &= \delta z_i + \nu_i \end{align} Here, everything has a zero mean, and $z$ is a valid instrument for $x$. For 2SLS and CF, we run auxilliary regressions like: \begin{align} y_i &= \beta_\text{2SLS} \hat{x}_i + \epsilon_{2i}\\ y_i &= \beta_\text{CF} x_i + \delta \hat{\nu}_i + \epsilon_{3i} \end{align} Now, the 2SLS estimator will be: \begin{align} \hat{\beta}_\text{2SLS} &= \frac{\sum y_i \hat{x}_i}{\sum (\hat{x}_i)^2} \end{align} By the Frisch-Waugh-Lovell Theorem, the CF estimator will be: \begin{align} \hat{\beta}_\text{CF} &= \frac{\sum e_{y|\hat{\nu},i} e_{x|\hat{\nu},i}}{\sum (e_{x|\hat{\nu},i})^2} \end{align} In that expression, $e_{y|\hat{\nu}}$ means the residuals from a regression of $y$ on $\hat{\nu}$.

If you regress $x$ on $\hat{\nu}$, then you get a coefficient of 1 and the residuals from that regression are $\hat{x}$. So, the denominators of the two fractions are the same. Also, this gives you that the second terms in the numerators are the same.

What happens when you regress $y$ on $\hat{\nu}$? Well, the residuals from that regression are: \begin{align} y-\left(\frac{\sum y\hat{\nu}}{\sum(\hat{\nu})^2}\right)\hat{\nu} \end{align}
So, the conclusion follows from the fact that $\sum \hat{\nu}\hat{x}=0$.

It should not be hard to generalize this to the case with an intercept and more right hand side variables.

Answer 2

Here is a more general argument for the multivariate case to compute $\widehat{\delta}_{2SLS}$:

1. Regress the regressors $Z$ on the instruments $X$ and save the residuals $\widetilde{Z}:=M_{X}Z=Z-X(X'X)^{-1}X'Z$.
2. Regress $y$ on $Z$ and $\widetilde{Z}$, $$ y=Z\widehat{\delta}+\widetilde{Z}\widehat{\theta}+\widehat{u} $$ Recall that the FWL theorem states that we can obtain subvectors of coefficients on variables of "interest" of a long regression by regressing the residuals of a regression of the dependent variable on the remaining ("non-interesting") explanatory variables on the residuals of a regression of the submatrix of interest on the remaining variables.

We thus use FWL in 2. to show that $\widehat{\delta}_{{2SLS}}=\widehat{\delta}$:

\begin{eqnarray*} \widehat{\delta}&=&(Z'M_{\widetilde{Z}}Z)^{-1}Z'M_{\widetilde{Z}}y\\ &=&(Z'(I-P_{\widetilde{Z}})Z)^{-1}Z'(I-P_{\widetilde{Z}})y \end{eqnarray*} Now, \begin{eqnarray*} P_{\widetilde{Z}}&=&M_{X}Z(Z'M_{X}'M_{X}Z)^{-1}Z'M_{X}\\ &=&M_{X}Z(Z'M_{X}Z)^{-1}Z'M_{X} \end{eqnarray*} so that $$ (I-P_{\widetilde{Z}})Z=Z-M_{X}Z(Z'M_{X}Z)^{-1}Z'M_{X}Z=Z-M_{X}Z=P_{X}Z $$ such that $$ \widehat{\delta}=(Z'P_{X}Z)^{-1}Z'P_{X}y $$