First stage of TSLS and the matrix of instruments W

Question

If we assume we have 2 equations and each equation contains the other dependent variable.

$y_1 = \beta_0 + \beta_1 y_2 + \beta_2 z_1 + u_1$

$y_2 = \alpha_0 + \alpha_1 y_1 + \alpha_2 z_2 + u_2$

For exact identification, we would need two instruments. Let's say these instruments are $z_3$ and $z_4$.

The first stage of TSLS is equal to the regression of the assumed endogenous variables $y_1$ and $y_2$ on the matrix of instruments $W$ which contains all the instruments and exogenous variables. So the columns of the matrix of instruments would look like

$W = [z_1, z_2, z_3, z_4]$

and the first stage of TSLS like

$y_1 = W\pi_1 + v_1$

$y_2 = W\pi_2 + v_2$

For identification, I need the significance of exact one of those instruments, i.e., of $z_3$ and $z_4$ in each of those reduced form equations.

My question is: does matrix $W$ indeed contain both instruments or do we regress onto $W_1=[z_1, z_2, z_3]$ and $W_2=[z_1, z_2, z_4]$ if we take $z_3$ as a instrument for $y_1$ and the instrument variable $z_4$ for the endogenous variable $z_4$?

Is that right that we would need at least 3 instruments if we observe something like?

$y_1 = \beta_0 + \beta_1 y_2 + \beta_2 y_3 + \beta_3 z_1 + u_1$

$y_2 = \alpha_0 + \alpha_1 y_1 + \alpha_2 y_3 + \alpha_3 z_2 + u_2$

$y_3 = \gamma_0 + \gamma_1 y_1 + \gamma_2 y_2 + \gamma_3 z_3 + u_3$

so that the matrix of instruments would look like

$W = [z_1, z_2, z_3, z_4, z_5, z_6]$

if $z_4, z_5$ and $z_6$ are our instruments.

Answer 1

It would be useful for you to think in terms of geometry of least squares, i.e., geometry of projections onto subspaces generated by the columns of the regressor matrices. The space of $Y$s is "dirty", as it is affected by endogeneity problems. With the instrumental variables approach, we try to find a "cleaner" space in which the regressors would be orthogonal to the errors. We do so by projecting everything into the space of the instruments, which are known to be orthogonal to the errors $\mathbf{u}$. Now, in that space, we can run a nice regression of $y|W$ (a projection onto the column space of $W$) on regressors$|W$ (a projection of the regressors on $W$, again). If you projected $y_1$ on one subset, and $y_2$ on another, you may end up with a weird lack of orthogonality because the projected regressor vectors will be spiking out in different spaces, so nothing really is guaranteed.

A more algebraic approach can be used, too: instead of the normal equations $$ \sum_i x_i(y_i - x_i'\beta) = 0 $$ you use the instrumented equations $$ \sum_i z_i(y_i - x_i'\beta) = 0 $$ where I dumped all the regressors into $x_i$. From this, it is very clear that you should multiply all $Y$s and all $X$s by the same set of instruments $W$. Econometricians call this extremum estimator approach; statisticians, $M$-estimation approach.

Davidson and MacKinnon's EIE discusses the geometry and asymptotics to some extent in chapter 7. Fumio Hayashi's Econometrics treats everything from GMM perspective, so the instrumental variables are approached in the second, more algebraic, way (Ch. 4). Takeshi Amemiya's Advanced Econometrics gives three different interpretations of 2SLS (section 7.3.6).

Update: the rank condition states that if the structural form is $$ \mathbf{y\Gamma} + \mathbf{z \Delta} + u = 0 $$ with $\gamma_{kk}=-1$, and restrictions for the first equation are $$\mathbf{R}_1 \mathbf{\beta}_1 = 0$$ where $$\mathbf{B} = \left( \begin{array}{c} \mathbf{\Gamma} \\ \mathbf{\Delta} \end{array} \right), $$ then the first equation is identified when $${\rm rank} \mathbf{R}_1 \mathbf{B} = G-1$$ where $G$ is the number of endogenous variables.

Ignoring the intercepts, we have, in your notation, $$ \mathbf{\Gamma} = \left( \begin{array}{ccc}-1 & \alpha_1 & \gamma_1 \\ \beta_1 & -1 & \gamma_2 \\ \beta_2 & \alpha_2 & -1 \end{array} \right), \quad \mathbf{\Delta}_3 = \left( \begin{array}{ccc} \beta_3 & 0 & 0 \\ 0 & \alpha_3 & 0 \\ 0 & 0 & \gamma_3 \end{array} \right) $$ with just three instruments. The restrictions for the first equation are $$ \mathbf{R}_1 = \left( \begin{array}{cccccc} 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & \end{array} \right) $$ describing the omitting coefficients for instruments $z_2$ and $z_3$. The matrix in question is $$ \mathbf{R}_1 \mathbf{B} = \left( \begin{array}{cc} \alpha_3 & 0 \\ 0 & \gamma_3 \end{array} \right ) $$ For this matrix to have rank 2, both $\alpha_3$ and $\gamma_3$ would have to be non-zero.

With six instruments, you will have $\mathbf{\Delta}_6$ that has $\mathbf{\Delta}_3$ in the top half, and zeroes padded at the bottom half; and also three more rows of $\mathbf{R}_1$ with unities in the corresponding columns for the last three rows. The matrix $\mathbf{R}_1 \mathbf{B}$ grows more zeroes at its bottom, respectively. So the substantive condition does not change: $\alpha_3$ and $\gamma_3$ have to be non-zero. However, if these extra instruments appeared in the equations for $y_2$ and $y_3$, we could shift the identification burden from $\alpha_3$ and $\gamma_3$ to the coefficients of the excluded instruments in other equations. So adding more instruments does not harm identification.