Weak instrument but 2nd stage significant

Question

My IV's 1st stage F-stat is 4.7 (considered weak by the rule of thumb). The 2nd-stage regression coefficient is significant. I wonder what is the best way to interpret such observations.

Answer 1

If it's overidentified IV (more instruments than endogeneous regressors), the second-stage $t$-test can be severely distorted when instruments are weak (see for example Stock & Yogo (2005) p. 28 about the $F$>10 rule). Recent research (Angrist & Kolesár, 2023), however, has shown that if you're using a single instrument, you can interpret your second-stage $t$-test fairly reliably regardless of the first-stage $F$-statistic unless there is extreme endogeneity.

Our setup has two free parameters: the correlation between structural and first-stage residuals (henceforth, “endogeneity”), and the population first-stage $F$-statistic. This fact lends itself to the construction of easily-interpreted rejection contours characterizing conventional second-stage $t$-tests and confidence interval coverage rates. We see, for example, that for endogeneity less than about 0.76, 95% confidence interval coverage is distorted by no more than 5% for any population $F$. This is explained by the fact that, even as median bias increases when the first stage gets weaker, second-stage precision falls (we focus on median bias because the conventional just-ID IV estimator has no moments). In contrast with the over-identified case, conventional just-ID IV standard errors reflect this, and confidence intervals widen accordingly. This fact keeps interval coverage high unless endogeneity is extraordinarily high. (Angrist & Kolesár, 2023)

So, if both the first stage and the reduced form are significant and your results are robust to whatever checks you have devised to assess their credibility, I wouldn't worry too much about $F$<10.

REFERENCES:

• Angrist, J., & Kolesár, M. (2023). One instrument to rule them all: The bias and coverage of just-ID IV. Journal of Econometrics. https://doi.org/10.1016/j.jeconom.2022.12.012
• Stock, J. H. and Yogo, M. (2005). Testing for Weak Instruments in Linear IV Regression. IDENTIFICATION AND INFERENCE FOR ECONOMETRIC MODELS: ESSAYS IN HONOR OF THOMAS ROTHENBERG. Available at SSRN: https://ssrn.com/abstract=1734933

Answer 2

Just for fun, the following is ChatGPT's response (not helpful repeating what we already know):

``When dealing with instrumental variables (IV) estimation, particularly in a two-stage least squares (2SLS) framework, the strength of the instrument is crucial for reliable inference. Your observations involve two key points: a weak first-stage F-statistic and a significant second-stage regression coefficient. Here's how to interpret these observations:

Weak First-Stage F-Statistic (4.7): The first stage of an IV/2SLS estimation involves regressing the endogenous variable on the instrumental variable(s). The F-statistic from this regression tests the null hypothesis that the instrument(s) are weak, i.e., they are not correlated with the endogenous explanatory variable. A rule of thumb is that the F-statistic should be greater than 10 to ensure that the instrument is not weak. An F-statistic of 4.7, as in your case, suggests that the instrument might be weak. This raises concerns about the reliability of the 2SLS estimates, as weak instruments can lead to biased and inconsistent parameter estimates and weaken the inference in the second stage.

Significant Second-Stage Regression Coefficient: Despite the weak first-stage F-statistic, you've found a significant coefficient in the second-stage regression, where the predicted values from the first stage are used as instruments for the endogenous variable. This significance suggests that, conditional on the instrument, there is a statistically significant relationship between the instrumented variable and the dependent variable. However, this result should be interpreted with caution due to the weak instrument issue in the first stage.

Given these observations, here's how to proceed:

Reassess the Instrument: Ensure that the instrument is theoretically valid, meaning it affects the dependent variable only through the endogenous explanatory variable and is not correlated with the error term of the outcome equation. If the theoretical justification is strong, the concern about weak instrumentation might be somewhat mitigated, but empirical concerns remain.

Use Alternative Instruments: If possible, consider finding and using alternative or additional instruments that are stronger (i.e., more correlated with the endogenous variable but still exogenous).

Robustness Checks: Conduct robustness checks using different specifications or subsets of the data to see if the results hold.

Report the Weak Instrument Issue: When presenting your results, it's crucial to discuss the potential limitations due to the weak first-stage F-statistic. This includes the potential for biased and inconsistent estimates.

Consider Other Methods: If the instrument's weakness is a persistent issue, consider other estimation strategies that might not rely on instrumental variables, such as natural experiments, regression discontinuity designs, or difference-in-differences approaches, if applicable to your research question and data.

Use Weak Instrument Robust Inference: There are statistical methods designed to deal with weak instruments, such as the Limited Information Maximum Likelihood (LIML) approach or the use of a Fuller adjustment. These methods can provide more reliable inference when the instruments are weak.

In summary, while a significant second-stage coefficient is encouraging, the weak first-stage F-statistic raises concerns about the reliability of the IV estimates. Careful consideration of the instrument's validity, exploration of alternative instruments, and robustness checks are essential steps in addressing this issue.''

Answer 3

Countering Jonathan's answer, I would recommend using a test that is robust to weak instruments to assess the significance of your estimate.

Examples of such tests are the Anderson-Rubin test, the conditional likelihood ratio test, and the (Kleibergen) Lagrange Multiplier test.