3
$\begingroup$

This question is related to Bias of more than one endogenous variables, and I am essentially asking for some help regarding proving the results of the answer algebraically along with some related questions.

Suppose I have the linear model $Y = X\beta + W\gamma$ + U, where $Y,X,W$ are all matrices, and $W$ is a set of controls. I have my instrument for $X$, the variable of interest. I do not know if $W$ is endogenous or not, but because it is just a bunch of controls, I naively figure I don't need to worry about them because I'm ONLY concerned with estimating $\beta$. I basically want the algebraic proof for why it does matter (which seems to be the case).

The Set Up:

Because $X$ may be endogenous, I perform the following 2SLS, with instrument $Z$ for $X$. The model would be as follows: $$\text{first stage: } X = Z\alpha + T\theta + V \\ \text{second stage: } Y =\hat{X}\beta + W\delta + D $$ where $T$ is a set of first stage controls, and $W$ is the set of second stage controls, and $\hat{X}$ is the fitted values of $X$. My questions are as follows (assuming I correctly set up the problem):

  1. I am correct that the error term $O$ is generally not the same as U in the true model, right?
  2. Does it matter if W, the set of controls for the second stage, are endogenous, if all I care about is the coefficient $\beta$? Based on the linked question, it does, but what is the algebraic proof? I have not been able to do much.
  3. Similar to 2. , but does it matter if T is endogenous, again if all I care about is estimating $\beta$?

$\textbf{Update}$: I did some work on this which I was able to check based on @semibruin's fantastic answer, and have the following results:

Considering the endogeneity problems, we have: \begin{align*} \hat{\beta}^{2sls}=&(\hat{X}'M_w\hat{X})^{-1}\hat{X}'M_wY\text{ (where }M_w=I-W(W'W)^{-1}W'\text{)} \\ =&(\hat{X}'M_w\hat{X})^{-1}\hat{X}'M_w(\hat{X}\beta+W\delta+D) \\ =&\beta+0+(\hat{X}'M_w\hat{X})^{-1}\hat{X}M_wD \\ \xrightarrow{plim}&\beta+E[\hat{X}'M_w\hat{X}]^{-1}E[\hat{X}'M_wD] \end{align*} So the estimator $\hat{\beta}^{2sls}$ is consistent when $E[\hat{X}'M_wD]=0$. Focusing on that expression, we look at one issue at a time:\ First, let's ignore the first controls (ie set $T=0$). Then $\hat{X}=Z\hat{\alpha}$ Recalling results from linear algebra and partitioned regressions, $P_w:=W(W'W)^{-1}W'\text{ and } M_w:=I-P_w$ are both projection matrices (in particular, it can be easily verified that $P_wW=W$ and $M_wW=0$), and we have $Z\hat{\alpha}=Z(Z'Z)^{-1}Z'X$. Furthermore, for any $X$, $X'X$ is a symmetric matrix (so that $(X'X)^{-1}{'}=(X'X)^{-1} $ ). Then, since $X'=\alpha'Z'+V'$, we have: \begin{align*} E[\hat{X}'M_wD] &=E[(Z\hat{\alpha})'M_wD] \\ &=E[(Z(Z'Z)^{-1}Z'X)'(I-W(W'W)^{-1}W')D]\\ &=E[X'Z(Z'Z)^{-1}Z'(I-W(W'W)^{-1}W')D]\\ &=E[X'P_zD]-E[X'P_zP_wD]\\ &=\alpha'E[Z'D]+E[V'P_ZD]-\alpha'E[Z'P_wD]-E[V'P_zP_wD]\\ &=E[V'P_ZD]-\alpha'E[Z'P_wD]-E[V'P_zP_wD]\\ &\text{and I'm not sure where to go next} \end{align*} Thus, we are not guaranteed consistency, and have consistency of $\hat{\beta}$ only if...

Next, let's ignore the second controls (ie set $W=0$), but consider the first controls. Thus, $\hat{X}=Z\hat{\alpha}+T\hat{\theta}$, $\hat{\beta}=(\hat{X}'\hat{X})^{-1}\hat{X}'Y$ where $Y=\hat{X}\beta+D$, and $\hat{\beta}^{2sls}\xrightarrow{plim}\beta+E[\hat{X}'\hat{X}]^{-1}E[\hat{X}'D]$. Then, recalling that $X=Z\alpha+T\theta+V$: \begin{align*} E[\hat{X}'D]&=E[(Z((Z'M_TZ)^{-1}Z'M_TX)+T((T'M_ZT)^{-1}T'M_ZX))'D]\\ &=E[(Z(Z'M_TZ)^{-1}Z'M_T(Z\alpha+T\theta+V))'D]+E[(T(T'M_ZT)^{-1}T'M_Z(Z\alpha+T\theta+V))'D]\\ &=E[Z\alpha)'D]+0+E[(Z(Z'M_TZ)^{-1}Z'M_TV)'D]+0+E[(T\theta)'D]+E[(T(T'M_ZT)^{-1}T'M_ZV)'D]\\ &=E[(Z(Z'M_TZ)^{-1}Z'M_TV)'D]+\theta E[T'D]+E[(T(T'M_ZT)^{-1}T'M_ZV)'D] \text{ (recall $E[T'D]\neq0$)}\\ &\text{again, I'm not sure where to go next} \end{align*} Again, we are not guaranteed consistency, and have consistency of $\hat{\beta}$ only if...

In the first case, it would seem that if $Cov[Z,W] =0$, then we would have consistency. In the second case, even if $Cov[Z,T]=0$, because $E[T'D]\neq0$, we will still not have consistency, and likewise for $Cov[Z,T]=0$ in the second case.

$\textbf{Update 2}: Responding to @semibruin's comments, here is some more work I have that explains the above formulations:

We have that $X=\hat{X}+\hat{V}$, so plugging this into the true model, we find that $Y=\hat{X}\beta_{1}+W\gamma+(\hat{V}\beta+U)$, so $D=(\hat{V}\beta+U) \text{ and thus }\beta_{1}=\beta$. \ \ Using this, and recalling that $Cov[Z'U]=E[Z'U]-E[Z]E[U]=E[Z'U]$ as $E[U]=0$, we have that \begin{align*} E[Z'D]&=E[Z'U]+E[Z\hat{V}\beta]\\ &=E[Z'U]+(E[Z'X]-E[Z'\hat{X}])\beta] \end{align*} Ignoring the control variables from the set-up momentarily, the model $X=Z\alpha+V$ gives $\hat{X}=Z\hat{\alpha}=Z(Z'Z)^{-1}Z'X$, and plugging this into the above, we get: \begin{align*} E[Z'D]&=E[Z'U]+(E[Z'X]-E[Z'Z(Z'Z)^{-1}Z'X])\beta\\ &=E[Z'U]+(E[Z'X]-E[Z'X])\\ &=E[Z'U] \end{align*} Yay! So this shows if either of the exogeneity requirements holds, then the other automatically does too.

$\endgroup$
1
$\begingroup$

Yes and no. If $x$ and $w$ are correlated, the endogeneity of $w$ will bias the estimation of $\beta$, partial effect of $x$.

Let $Y$, $X$, $\hat{X}$ and $W$ be the matrices by stacking the observations of $y$, $x$ et al. The second stage OLS then equals \begin{equation*} \hat{\beta} = (\hat{X}'Q\hat{X})^{-1}\hat{X}'QY, \end{equation*} where $Q = I - W(W'W)^{-1}W'$. This follows the standard partitioned regression result.

Note $Y = X\beta + W\gamma + U$. We can write \begin{align*} \hat{\beta} &= (\hat{X}'Q\hat{X})^{-1}\hat{X}'Q(X\beta + W\gamma + U) \\ &= \beta + (\hat{X}'Q\hat{X})^{-1}\hat{X}'QW\gamma + (\hat{X}'Q\hat{X})^{-1}\hat{X}'QU \\ &= \beta + (\hat{X}'Q\hat{X})^{-1}\hat{X}'QU. \end{align*} The last line follows from $QW = 0$. The consistency then requires that the second term of the last line goes to zero. We have \begin{equation*} (\hat{X}'Q\hat{X})^{-1}\hat{X}'QU = (n^{-1}\hat{X}'Q\hat{X})^{-1}n^{-1}\hat{X}'QU, \end{equation*} where $n$ is sample size. The LLN will make $n^{-1}\hat{X}'Q\hat{X} \rightarrow_{p} A$ for some matrix $A$. The consistency requires \begin{equation*} n^{-1}\hat{X}'QU \rightarrow_{p} 0, \end{equation*} which in general is not true.

To explain, replacing $Q$ with its definition, we have \begin{equation*} n^{-1}\hat{X}'(I - W(W'W)^{-1}W')U = n^{-1}\hat{X}'U - n^{-1}\hat{X}'W(W'W)^{-1}W'U. \end{equation*} Of course, $n^{-1}\hat{X}'U = E(\hat{x}_{i}u_{i}) = 0$ by the first stage IV. This relates to your third question. If one instrument, $M$ in your notation, is endogenous, $\hat{X}$ will still be endogenous, $E(\hat{x}u) \neq 0$, and $\hat{\beta}$ will not be consistent.

Note that $(W'W)^{-1}W'U$ is the OLS estimator of regression $u$ on $w$, whose probability limit is \begin{equation*} \delta = (E(ww'))^{-1}E(wu). \end{equation*} Hence, \begin{equation*} n^{-1}\hat{X}'W(W'W)^{-1}W'U \rightarrow_{p} n^{-1}\hat{X}'W\delta = E(\hat{x}w')\delta. \end{equation*} If $w$ is endogenous, i.e. $E(wu) \neq 0$, $\delta \neq 0$. Unless $E(\hat{x}w') = 0$, the above display is not zero, and $\hat{\beta} -\beta \not\rightarrow_{p} 0$.

$\endgroup$
  • $\begingroup$ Thanks @semibruin. I updated my work in light of your amazing feedback, and have a couple additional questions (see my post), specifically relating to whether you had any thoughts about how I could proceed in my work. Regardless, what you wrote is fantastic, and really helps me understand this! $\endgroup$ – doubled Jun 29 '18 at 23:12
  • 1
    $\begingroup$ @doubled I would write $Y = X\beta + W\gamma + U$, rather than $Y = \hat{X}\beta + W \gamma + D$. Writing $Y = X\beta + W\gamma + U$ inserts $U$ into your arguments. This is helpful, since endogeneity is defined as the correlation between $U$ and explanatory variables. Using $U$ makes the idea more transparent. Also $Y = \hat{X}\beta + W \gamma + D$ is unclear and not necessarily true. What is $D$ anyway? I think in your mind, you want to write $Y$ as a linear projection of $\hat{X}$ and $W$, i.e. $Y = X b + W c + V$, but $b$ and $c$ are different from $\beta$ and $\gamma$ without exogeneity. $\endgroup$ – semibruin Jun 30 '18 at 4:04
  • $\begingroup$ @doubled In case you are unfamiliar with 'linear projection' in econometrics, check out the first few chapters of Jeff Wooldridge's book 'cross-seciton and panel data analysis'. $\endgroup$ – semibruin Jun 30 '18 at 4:06
  • $\begingroup$ I updated my post to reflect your questions. I am aware of linear projections, but doesn't my work I just wrote up show that that $\beta$ = b and, analogously, $\gamma$ = c (using the parameters from your example? Or did I make a mistake somewhere? The reason I use $\hat{X}$ is because i was under the impression that that is required for 2sls? $\endgroup$ – doubled Jun 30 '18 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.