Why is the variance of 2SLS bigger than that of OLS?

... Another potential problem with applying 2SLS and other IV procedures is that the 2SLS standard errors have a tendency to be ‘‘large.’’ What is typically meant by this statement is either that 2SLS coefficients are statistically insignificant or that the 2SLS standard errors are much larger than the OLS standard errors. Not surprisingly, the magnitudes of the 2SLS standard errors depend, among other things, on the quality of the instrument(s) used in estimation.

This quote is from Wooldridge's "Econometric analysis of cross-sectional and panel data". I wonder why this happens? I would prefer a mathematical explanation.

Assuming homoskedasticity for simplicity the (estimated) asymptotic variance of OLS estimator is given by $$\widehat{Avar}(\hat{\beta}_{OLS}) = n\sigma^2(X'X)^{-1}$$ while for the 2SLS estimator $$\widehat{Avar}(\hat{\beta}_{2SLS}) = n\sigma^2(\hat{X}'\hat{X})^{-1}$$ where $$\hat{X} = P_zX = Z(Z'Z)^{-1}Z'X.$$

$X$ is the matrix of regressors, including the endogenous ones, and $Z$ is the matrix of instrumental variables.

So rewriting the variance for 2SLS gives $$\widehat{Avar}(\hat{\beta}_{2SLS}) = n\sigma^2\left(X'Z(Z'Z)^{-1}Z'X\right)^{-1}.$$

However, I can't conclude from above formulas that $\widehat{Avar}(\hat{\beta}_{2SLS}) \geq \widehat{Avar}(\hat{\beta}_{OLS})$.

• I think you forgot to take the inverse in your last expression of Avar of 2SLS. Oct 27, 2016 at 15:12
• You are right, corrected. Oct 27, 2016 at 17:25
• I made a few small edits, in particular regarding the definition of $Z$. Please check. Oct 28, 2016 at 7:03

We say a matrix $A$ is at least as large as $B$ if their difference $A-B$ is positive semidefinite (psd).

An equivalent statement that turns out to be handier to check here is that $B^{-1}-A^{-1}$ is psd (much like $a>b$ is equivalent to $1/b>1/a$).

So we want to check that $$X'X-X'Z(Z'Z)^{-1}Z'X$$ is psd.

Write $$X'X-X'Z(Z'Z)^{-1}Z'X=X'(I-Z(Z'Z)^{-1}Z')X=X'M_ZX$$ To check that $X'M_ZX$ is psd, we must show that, for any vector $d$, $$d'X'M_ZXd\geq0$$ Let $c=Xd$. Then, $$c'M_Zc\geq0$$ as $M_Z$ is a symmetric and idempotent projection matrix, which is known to be psd: write, using symmetry and idempotency, $$c'M_Zc=c'M_ZM_Zc=c'M_Z'M_Zc$$ and let $e=M_Zc$, so that $c'M_Zc=e'e=\sum_ie_i^2$, which, being a sum of squares, must be nonnegative.

P.S.: Two little quibbles - you refer to the estimated asymptotic variances $\widehat{Avar}(\hat\beta_j)$. Now, the OLS estimator and the 2SLS estimator of $\sigma^2$ are not the same, so that I do not see that the ranking must necessarily be preserved if these estimates differ. Also, the asymptotic variances are generally scaled by $n$ so as to obtain a nondegenerate quantity as $n\to\infty$. (Of course, scaling both by $n$ will not affect the ranking, so that the issue is a little moot for this particular question.)

• Many thanks for your answer. Asymptotic variance indeed should be divided by $n$ (corrected). I guess there is a typo when you call $M_z$ projection matrix, I think it is called annihilator matrix. Anyway, could you please provide details why $M_z$ is psd. I also don't quite understand your point that OLS and 2SLS estimators for $\sigma^2$ are not the same, could you elaborate what it means? Oct 28, 2016 at 8:31
• I added some detail. $M$ is indeed best known as the annihilator matrix, but since it also projects on some space (the orthogonal complement of the image of $P$) it is also a projection matrix. Oct 28, 2016 at 8:38
• Thanks for clarification and edits (I don't know why I decided to divide by $n$). Could you explain your first point in P.S? Oct 28, 2016 at 8:49
• To really make it the estimated asymptotic variance, you would need some estimator $\hat\sigma^2$. The OLS estimator of $\sigma^2$ is based on the OLS residuals, whereas the 2SLS estimator uses residuals $y-X\hat\beta_{2SLS}$ to estimate $\sigma^2$. These estimates may differ, in either direction, possibly affecting the ranking of the variances. Oct 28, 2016 at 8:57

I think this is one those times where it is much easier to look at the simple one equation, one variable setting. So tehcnically this is IV-regression and not 2SLS (but the result is still general). So we will asume a model (using Wooldridge notation), for some $i$ we have:

$$y_i = \beta_0 + \beta_1 x_{i1} + u_i$$

Now, if we assume that this models follows the Gauss-Markov assumptions then we know (see any decent textbook) that the asymptotic variance of $\hat\beta_1$ is given by:

$$Avar(\hat\beta_{OLS})=\frac{\hat\sigma^2}{SST_x}$$

Where $SST_x$ is the total sum of squares for $x$. If instead we assume that $x$ is (possible) endegonoues, and use IV regression with $z$ as an instrument, then the asymptotic variance of the IV estimator is:

$$Avar(\hat\beta_{iv}) = \frac{\hat\sigma^2}{SST_x \cdot R^2_{x,z}}$$

Since $R^2$ is always between $0$ and $1$, it must be the case that the denominator for the IV estimator is smaller then for OLS (if OLS is actually valid).

Just a comment. I guess that it is pretty clear that the estimate of the variance of the errors is higher when using 2SLS. Recall that OLS minimizes the estimate of this variance. So, any other estimator should have a higher sample estimate of the variance of the errors.