# First stage of TSLS and the matrix of instruments W

If we assume we have 2 equations and each equation contains the other dependent variable.

$y_1 = \beta_0 + \beta_1 y_2 + \beta_2 z_1 + u_1$

$y_2 = \alpha_0 + \alpha_1 y_1 + \alpha_2 z_2 + u_2$

For exact identification, we would need two instruments. Let's say these instruments are $z_3$ and $z_4$.

The first stage of TSLS is equal to the regression of the assumed endogenous variables $y_1$ and $y_2$ on the matrix of instruments $W$ which contains all the instruments and exogenous variables. So the columns of the matrix of instruments would look like

$W = [z_1, z_2, z_3, z_4]$

and the first stage of TSLS like

$y_1 = W\pi_1 + v_1$

$y_2 = W\pi_2 + v_2$

For identification, I need the significance of exact one of those instruments, i.e., of $z_3$ and $z_4$ in each of those reduced form equations.

My question is: does matrix $W$ indeed contain both instruments or do we regress onto $W_1=[z_1, z_2, z_3]$ and $W_2=[z_1, z_2, z_4]$ if we take $z_3$ as a instrument for $y_1$ and the instrument variable $z_4$ for the endogenous variable $z_4$?

Is that right that we would need at least 3 instruments if we observe something like?

$y_1 = \beta_0 + \beta_1 y_2 + \beta_2 y_3 + \beta_3 z_1 + u_1$

$y_2 = \alpha_0 + \alpha_1 y_1 + \alpha_2 y_3 + \alpha_3 z_2 + u_2$

$y_3 = \gamma_0 + \gamma_1 y_1 + \gamma_2 y_2 + \gamma_3 z_3 + u_3$

so that the matrix of instruments would look like

$W = [z_1, z_2, z_3, z_4, z_5, z_6]$

if $z_4, z_5$ and $z_6$ are our instruments.

-

It would be useful for you to think in terms of geometry of least squares, i.e., geometry of projections onto subspaces generated by the columns of the regressor matrices. The space of $Y$s is "dirty", as it is affected by endogeneity problems. With the instrumental variables approach, we try to find a "cleaner" space in which the regressors would be orthogonal to the errors. We do so by projecting everything into the space of the instruments, which are known to be orthogonal to the errors $\mathbf{u}$. Now, in that space, we can run a nice regression of $y|W$ (a projection onto the column space of $W$) on regressors$|W$ (a projection of the regressors on $W$, again). If you projected $y_1$ on one subset, and $y_2$ on another, you may end up with a weird lack of orthogonality because the projected regressor vectors will be spiking out in different spaces, so nothing really is guaranteed.

A more algebraic approach can be used, too: instead of the normal equations $$\sum_i x_i(y_i - x_i'\beta) = 0$$ you use the instrumented equations $$\sum_i z_i(y_i - x_i'\beta) = 0$$ where I dumped all the regressors into $x_i$. From this, it is very clear that you should multiply all $Y$s and all $X$s by the same set of instruments $W$. Econometricians call this extremum estimator approach; statisticians, $M$-estimation approach.

Davidson and MacKinnon's EIE discusses the geometry and asymptotics to some extent in chapter 7. Fumio Hayashi's Econometrics treats everything from GMM perspective, so the instrumental variables are approached in the second, more algebraic, way (Ch. 4). Takeshi Amemiya's Advanced Econometrics gives three different interpretations of 2SLS (section 7.3.6).

Update: the rank condition states that if the structural form is $$\mathbf{y\Gamma} + \mathbf{z \Delta} + u = 0$$ with $\gamma_{kk}=-1$, and restrictions for the first equation are $$\mathbf{R}_1 \mathbf{\beta}_1 = 0$$ where $$\mathbf{B} = \left( \begin{array}{c} \mathbf{\Gamma} \\ \mathbf{\Delta} \end{array} \right),$$ then the first equation is identified when $${\rm rank} \mathbf{R}_1 \mathbf{B} = G-1$$ where $G$ is the number of endogenous variables.

Ignoring the intercepts, we have, in your notation, $$\mathbf{\Gamma} = \left( \begin{array}{ccc}-1 & \alpha_1 & \gamma_1 \\ \beta_1 & -1 & \gamma_2 \\ \beta_2 & \alpha_2 & -1 \end{array} \right), \quad \mathbf{\Delta}_3 = \left( \begin{array}{ccc} \beta_3 & 0 & 0 \\ 0 & \alpha_3 & 0 \\ 0 & 0 & \gamma_3 \end{array} \right)$$ with just three instruments. The restrictions for the first equation are $$\mathbf{R}_1 = \left( \begin{array}{cccccc} 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & \end{array} \right)$$ describing the omitting coefficients for instruments $z_2$ and $z_3$. The matrix in question is $$\mathbf{R}_1 \mathbf{B} = \left( \begin{array}{cc} \alpha_3 & 0 \\ 0 & \gamma_3 \end{array} \right )$$ For this matrix to have rank 2, both $\alpha_3$ and $\gamma_3$ would have to be non-zero.

With six instruments, you will have $\mathbf{\Delta}_6$ that has $\mathbf{\Delta}_3$ in the top half, and zeroes padded at the bottom half; and also three more rows of $\mathbf{R}_1$ with unities in the corresponding columns for the last three rows. The matrix $\mathbf{R}_1 \mathbf{B}$ grows more zeroes at its bottom, respectively. So the substantive condition does not change: $\alpha_3$ and $\gamma_3$ have to be non-zero. However, if these extra instruments appeared in the equations for $y_2$ and $y_3$, we could shift the identification burden from $\alpha_3$ and $\gamma_3$ to the coefficients of the excluded instruments in other equations. So adding more instruments does not harm identification.

-
Well I think I mixed things up but the cloud is still not completely gone. If I look at the rank condition for identification of a structual equation (see Wooldridge, chapter 16) then I can identify the $y_1$ and $y_2$ equation if $z_2$ (excluded from the 1st equation) has a non zero coefficient in the reduced form of $y_1$. Vice versa I can identify $y_2$ if $z_1$ (excluded from the 2nd equation) has a non zero coefficient in the reduced form of $y_2$. Since both reduced form equations contain $z_1$ and $z_2$ the matrix of instruments $W$ is equal. –  Druss2k Dec 13 '12 at 16:03
That's actually somewhat more advanced thinking that my answer was aimed at, as I did not even approach identification issues. And yes, you need to follow the rank and order conditions and have enough excluded instruments to make things work. My reading of the rank condition, though, is that $z_2$ must have a non-zero coefficient in the structural form for $y_2$; or, with more instruments, that at least one of the instruments excluded from the first equation has a non-zero coefficient in the structural form for the second. Or something like that. –  StasK Dec 13 '12 at 16:45
Yes $z_2$ and $z_1$ need to have a non zero coefficient in the structual equations. But with regards to the TSLS-estimation procedure we can use the variable $z_2$ as an instrument for $y_2$ since $z_2$ does not appear in the $y_1$ equation. If we do that and rewrite both depended variables in terms of their reduced form equations the variable $z_2$ needs to have a non zero coefficient does it not? I mean since $z_2$ is exogenous and it does have a impact on $y_2$ it will, if the $y_1$ model is actually correct specified, only have a indirect influence on $y_1$ namely throught $y_2$. –  Druss2k Dec 13 '12 at 16:54
The reduced form gets convoluted by multiplication of $\mathbf{\Gamma}^{-1}$, so I would be very cautious making any statements about the reduced form. If nothing is significant in the reduced form, you are in an obvious trouble of weak instruments. Unless you can derive the reduced form explicitly, there is little telling if a zero/non-significant coefficient in the reduced form is troublesome. –  StasK Dec 13 '12 at 17:11
Yes. I Know that the specification of a just identified model is a big source of trouble. But in this example I was not even sure if the model can actually be identified with only two exclusion restrictions in each equation. In my study I dont really need to find a good instruments since i'm quite convinced that there are non^^. By that i'll get the weak instrument issue anyway. Since i'm using a different approach I just wanted to state the potentiall troubles with a TSLS-Model. –  Druss2k Dec 13 '12 at 17:24