Why is the variance of 2SLS bigger than that of OLS?

Question

... Another potential problem with applying 2SLS and other IV procedures is that the 2SLS standard errors have a tendency to be ‘‘large.’’ What is typically meant by this statement is either that 2SLS coefficients are statistically insignificant or that the 2SLS standard errors are much larger than the OLS standard errors. Not surprisingly, the magnitudes of the 2SLS standard errors depend, among other things, on the quality of the instrument(s) used in estimation.

This quote is from Wooldridge's "Econometric analysis of cross-sectional and panel data". I wonder why this happens? I would prefer a mathematical explanation.

Assuming homoskedasticity for simplicity the (estimated) asymptotic variance of OLS estimator is given by $$\widehat{Avar}(\hat{\beta}_{OLS}) = n\sigma^2(X'X)^{-1}$$ while for the 2SLS estimator $$\widehat{Avar}(\hat{\beta}_{2SLS}) = n\sigma^2(\hat{X}'\hat{X})^{-1}$$ where $$\hat{X} = P_zX = Z(Z'Z)^{-1}Z'X.$$

$X$ is the matrix of regressors, including the endogenous ones, and $Z$ is the matrix of instrumental variables.

So rewriting the variance for 2SLS gives $$\widehat{Avar}(\hat{\beta}_{2SLS}) = n\sigma^2\left(X'Z(Z'Z)^{-1}Z'X\right)^{-1}.$$

However, I can't conclude from above formulas that $\widehat{Avar}(\hat{\beta}_{2SLS}) \geq \widehat{Avar}(\hat{\beta}_{OLS})$.

Answer 1

We say a matrix $A$ is at least as large as $B$ if their difference $A-B$ is positive semidefinite (psd).

An equivalent statement that turns out to be handier to check here is that $B^{-1}-A^{-1}$ is psd (much like $a>b$ is equivalent to $1/b>1/a$).

So we want to check that $$ X'X-X'Z(Z'Z)^{-1}Z'X $$ is psd.

Write $$ X'X-X'Z(Z'Z)^{-1}Z'X=X'(I-Z(Z'Z)^{-1}Z')X=X'M_ZX $$ To check that $X'M_ZX$ is psd, we must show that, for any vector $d$, $$ d'X'M_ZXd\geq0 $$ Let $c=Xd$. Then, $$ c'M_Zc\geq0 $$ as $M_Z$ is a symmetric and idempotent projection matrix, which is known to be psd: write, using symmetry and idempotency, $$ c'M_Zc=c'M_ZM_Zc=c'M_Z'M_Zc $$ and let $e=M_Zc$, so that $c'M_Zc=e'e=\sum_ie_i^2$, which, being a sum of squares, must be nonnegative.

P.S.: Two little quibbles - you refer to the estimated asymptotic variances $\widehat{Avar}(\hat\beta_j)$. Now, the OLS estimator and the 2SLS estimator of $\sigma^2$ are not the same, so that I do not see that the ranking must necessarily be preserved if these estimates differ. Also, the asymptotic variances are generally scaled by $n$ so as to obtain a nondegenerate quantity as $n\to\infty$. (Of course, scaling both by $n$ will not affect the ranking, so that the issue is a little moot for this particular question.)

Answer 2

I think this is one those times where it is much easier to look at the simple one equation, one variable setting. So tehcnically this is IV-regression and not 2SLS (but the result is still general). So we will asume a model (using Wooldridge notation), for some $i$ we have:

$$ y_i = \beta_0 + \beta_1 x_{i1} + u_i $$

Now, if we assume that this models follows the Gauss-Markov assumptions then we know (see any decent textbook) that the asymptotic variance of $\hat\beta_1$ is given by:

$$ Avar(\hat\beta_{OLS})=\frac{\hat\sigma^2}{SST_x} $$

Where $SST_x$ is the total sum of squares for $x$. If instead we assume that $x$ is (possible) endegonoues, and use IV regression with $z$ as an instrument, then the asymptotic variance of the IV estimator is:

$$ Avar(\hat\beta_{iv}) = \frac{\hat\sigma^2}{SST_x \cdot R^2_{x,z}} $$

Since $R^2$ is always between $0$ and $1$, it must be the case that the denominator for the IV estimator is smaller then for OLS (if OLS is actually valid).

Answer 3

Just a comment. I guess that it is pretty clear that the estimate of the variance of the errors is higher when using 2SLS. Recall that OLS minimizes the estimate of this variance. So, any other estimator should have a higher sample estimate of the variance of the errors.