Let us work with the following structural model: $$y=\mathbf{x_{1i}β}+x_{2i}β_2+\varepsilon_i$$

where $x_{2i}$ is our single endogenous regressor. It turns out that given my instruments and my first stage regression, I can obtain $β_2$ in two different ways. I can use the two stage least squares regression $$y=\mathbf{x_{1i}β}+\hat{x}_{2i}β_2+\varepsilon_i \qquad (*)$$ where $\hat{x}_{2i}$ is our instrument predicted with the first stage regression.

And we can obtain the same estimate for $β_2$ with the following regression $$y=\mathbf{x_{1i}\beta}+x_{2i}β_2+\delta\hat{v}_i+\varepsilon_i \qquad (**)$$

where $\hat{v}_i$ is the estimated residual of the first stage regression, namely the portion of the original endogenous regressor that is correlated with the original $\varepsilon_i$.

Now I would like to prove algebraically that we can estimate $β_2$ using these two different approaches. I understand why the estimates must be the same. After all, in $(*)$ we are estimating the partial effect of $x_{2i}$ on $y$ controlling for the portion of $x_{2i}$ that was correlated with the error in the structural model above. Is this interpretation correct? Furthermore I tried to explicit this algebraically. I know I should be able to rewrite $(*)$ as $(**)$ or viceversa. Unfortunately I am not convinced of my result. I started from $(*)$, used the fact that $\hat{v_i}=x_{2i}-\hat{x}_{2i}$ and got the following result $$y=\mathbf{x_{1i}β}+x_{2i}β_2-β_2\hat{v_i}+\varepsilon_i$$

Now, this result is similar to $(**)$ but I cannot understand it. In particular, how should I decompose $x_{2i}$? What am I missing? Could you give me a hint?

Thank you for your help.

  • 2
    $\begingroup$ Given that $x_{2i} = \widehat{x}_{2i} + \widehat{\nu}_i$, it seems more intuitive to me to substitute that into the structural equation at the beginning. In the last equation you have $\widehat{\nu}$ and $x_{2i}$ but $\widehat{\nu}_i$ is already a part of $x_{2i}$. The procedure of course works for the point estimates but the standard errors will be completely wrong and cannot be saved by bootstrapping, for instance. You would also need to derive the variance estimator and correct by hand in that case. $\endgroup$ – Andy Dec 2 '14 at 22:17
  • $\begingroup$ Now I would like to prove algebraically that we can estimate $\beta_2$ using these two different approaches. What exactly do you mean by that? Do you want to prove that the two point estimates will be equal, or something else? $\endgroup$ – Richard Hardy Dec 2 '14 at 22:21
  • $\begingroup$ @Richard Yes, I would like to prove that the point estimates are the same. I still cannot understand how. I tried to substitute $x_{2i}$ in the structral equation but I cannot end up with the equivalence I am looking for. What am I misunderstanding? $\endgroup$ – Charlie Dec 3 '14 at 6:15
  • 3
    $\begingroup$ @Charlie, what you are trying to do is referred to as the control function approach. The proof of equivalence between the CF estimator and the 2SLS estimator is somewhat more elaborate than what you start with as it relies on projection theorems. $\endgroup$ – Andy Dec 3 '14 at 19:41

When I am trying to figure out why a result is true, I usually look at it in the simplest possible case. So, let's try a bivariate system. Here is the structural system: \begin{align} y_i &= \beta x_i + \epsilon_i\\ x_i &= \delta z_i + \nu_i \end{align} Here, everything has a zero mean, and $z$ is a valid instrument for $x$. For 2SLS and CF, we run auxilliary regressions like: \begin{align} y_i &= \beta_\text{2SLS} \hat{x}_i + \epsilon_{2i}\\ y_i &= \beta_\text{CF} x_i + \delta \hat{\nu}_i + \epsilon_{3i} \end{align} Now, the 2SLS estimator will be: \begin{align} \hat{\beta}_\text{2SLS} &= \frac{\sum y_i \hat{x}_i}{\sum (\hat{x}_i)^2} \end{align} By the Frisch-Waugh-Lovell Theorem, the CF estimator will be: \begin{align} \hat{\beta}_\text{CF} &= \frac{\sum e_{y|\hat{\nu},i} e_{x|\hat{\nu},i}}{\sum (e_{x|\hat{\nu},i})^2} \end{align} In that expression, $e_{y|\hat{\nu}}$ means the residuals from a regression of $y$ on $\hat{\nu}$.

If you regress $x$ on $\hat{\nu}$, then you get a coefficient of 1 and the residuals from that regression are $\hat{x}$. So, the denominators of the two fractions are the same. Also, this gives you that the second terms in the numerators are the same.

What happens when you regress $y$ on $\hat{\nu}$? Well, the residuals from that regression are: \begin{align} y-\left(\frac{\sum y\hat{\nu}}{\sum(\hat{\nu})^2}\right)\hat{\nu} \end{align}
So, the conclusion follows from the fact that $\sum \hat{\nu}\hat{x}=0$.

It should not be hard to generalize this to the case with an intercept and more right hand side variables.


Here is a more general argument for the multivariate case to compute $\widehat{\delta}_{2SLS}$:

  1. Regress the regressors $Z$ on the instruments $X$ and save the residuals $\widetilde{Z}:=M_{X}Z=Z-X(X'X)^{-1}X'Z$.
  2. Regress $y$ on $Z$ and $\widetilde{Z}$, $$ y=Z\widehat{\delta}+\widetilde{Z}\widehat{\theta}+\widehat{u} $$ Recall that the FWL theorem states that we can obtain subvectors of coefficients on variables of "interest" of a long regression by regressing the residuals of a regression of the dependent variable on the remaining ("non-interesting") explanatory variables on the residuals of a regression of the submatrix of interest on the remaining variables.

We thus use FWL in 2. to show that $\widehat{\delta}_{{2SLS}}=\widehat{\delta}$:

\begin{eqnarray*} \widehat{\delta}&=&(Z'M_{\widetilde{Z}}Z)^{-1}Z'M_{\widetilde{Z}}y\\ &=&(Z'(I-P_{\widetilde{Z}})Z)^{-1}Z'(I-P_{\widetilde{Z}})y \end{eqnarray*} Now, \begin{eqnarray*} P_{\widetilde{Z}}&=&M_{X}Z(Z'M_{X}'M_{X}Z)^{-1}Z'M_{X}\\ &=&M_{X}Z(Z'M_{X}Z)^{-1}Z'M_{X} \end{eqnarray*} so that $$ (I-P_{\widetilde{Z}})Z=Z-M_{X}Z(Z'M_{X}Z)^{-1}Z'M_{X}Z=Z-M_{X}Z=P_{X}Z $$ such that $$ \widehat{\delta}=(Z'P_{X}Z)^{-1}Z'P_{X}y $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.