How to interpret second-stage coefficient in instrumental variables regression with a binary instrument and a binary endogenous variable?

Question

(fairly long post, sorry. It includes lots of background info, so feel free to skip to the question at the bottom.)

Intro: I am working on a project where we are trying to identify the effect of a binary endogenous variable, $x_1$, on a continuous outcome, $y$. We have come up with an instrument, $z_1$, that we strongly believe to be as-if-randomly assigned.

Data: The data itself is in a panel structure with about 34,000 observations spread across 1000 units and about 56 time periods. $x_1$ takes on a value of 1 for about 700 (2 %) of the observations, and $z_1$ does so for about 3000 (9 %). 111 (0.33 %) observations score a 1 on both $z_1$ and on $x_1$, and it is twice as likely for an observation to score a 1 on $x_1$ if it also scores a 1 on $z_1$.

Estimation: We estimate the following 2SLS model through Stata’s ivreg2-procedure:

$$x_1 = \pi_0 + \pi_1z_1 + \mathbf{Z}\mathbf{\pi} + v$$ $$y = \beta_0 + \beta_1 x_1^* + \mathbf{Z}\mathbf{\beta} + u$$

Where $Z$ is a vector of other exogenous variables, $x_1^*$ is the predicted value of $x_1$ from the first stage, and $u$ and $v$ are error terms.

Results: Everything seems to be working well; the estimate of $\pi_1$ is highly significant in the first stage and the estimate of $\beta_1$ is highly significant in the second stage. All signs are as expected, including the ones for the other exogenous variables. The problem is, however, that the estimate of $\beta_1$ – the coefficient of interest – is implausible large (or, at least, according to the way we’ve been interpreting it).

$y$ ranges from about 2 to about 26 with mean and median of 17, but the estimate of $\beta_1$ ranges from 30 to 40 (depending on specification)!

Weak IV: Our first thought was that this was due to the instrument being too weak; that is, not correlated very much with the endogenous variable, but this does not really seem to be the case. To inspect the weakness of the instrument, we use Finlay, Magnusson, and Schaffer’s weakiv-package, as it provides tests that are robust to violations of the $i.i.d.$ assumption (which is relevant here, given that we have panel data and cluster our SE’s at the unit level).

According to their AR-test, the lower bound of the 95 % confidence interval for the second-stage coefficient is between 16 and 29 (again depending on specification). Rejection probability is practically 1 for all values anywhere close to zero.

Influential observations: We’ve tried estimating the model with each unit removed individually, with each observation removed individually, and with clusters of units removed. No real change.

Proposed solution: Someone proposed that we shouldn’t summarize the estimated effect of the instrumented $x_1$ in its original metric (0-1), but in the metric of its predicted version. $x_1^*$ ranges from -0.01 to 0.1 with mean and median of about 0.02 and an SD of about 0.018. If we were to summarize the estimated effect of $x_1$ by, say, a one SD increase in $x_1^*$, that would be $0.018*30 = 0.54$ (other specifications give almost identical results). This would be way more reasonable (yet still substantial). Seems like the perfect solution. Except I’ve never seen anybody do that; everybody just appears to interpret the second-stage coefficient using the metric of the original endogenous variable.

Question: In an IV-model, is it correct to summarize the estimated effect (the LATE, really) of an increase in the endogenous variable by using the metric of the predicted version of it? In our case, that metric is predicted probability.

Note: We use 2SLS even though we have a binary endogenous variable (making the first stage an LPM). This follows Angrist & Krueger (2001): “Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments”) We’ve also tried the three-stage procedure used in Adams, Almeida, & Ferreira (2009): “Understanding the relationship between founder–CEOs and firm performance”. The latter approach, which consists of a probit model followed by 2SLS, yields smaller and more sensible coefficients, but they are still very large if interpreted in the 0-1 metric (about 9-10). We get the same results with manual calculations as we do with the probit-2sls-option in Cerulli’s ivtreatreg.

Answer 1

This is an old question, but for anyone who stumbles across it in the future, intuitively the 2SLS estimate of $\beta_1$ is $\alpha_1$ from the "reduced form" regression

$$y = \alpha_0 + \alpha_1 z_1 + \mathbf{Z}\mathbf{\alpha} + u$$

divided by $\pi_1$ from the "first stage" regression

$$x_1 = \pi_0 + \pi_1z_1 + \mathbf{Z}\mathbf{\pi} + v$$

So if the 2SLS estimates of $\beta_1$ are "implausibly large," check the OLS estimates of $\alpha_1$ and $\pi_1$.

If the $\alpha_1$ estimates are "reasonable," the problem could be that the $\pi_1$ estimates are "very small." Dividing $\hat{\alpha}_1$ by a "very small" $\hat{\pi}_1$ can produce an "implausibly large" $\hat{\beta}_1$.