# What is next after a referee rejects an instrumental variable strategy?

I have a paper that just got rejected. It appears that my identification strategy is not sound enough. Below I describe the critique and seek some advice.

My sample is elderly aged 55 and over. The dependent is a health outcome. Two independent variables of interest are part-time (working less than 35 hours a week) and full-time work dummies, and hence the base is retired. Since health can affect work decisions (simultaneity), I take an IV approach. As instruments for working part-time and full-time, I use dummies indicating whether the individual has reached age 62 or 65 which are eligibility ages to receive early and normal social security benefits. I also consider age 70 for some reason I do not need to explain. I also consider the same eligibility ages for the partner with the argument being that partner's retirement status could affect the work decisions of the individual. Hence, in total I have six instruments. There is a literature analyzing the effects of retirement on health outcomes using eligibility ages as instruments for the retirement decision. In this literature the base outcome is working any number of hours. Hence, my idea is instead to differentiate between part-time and full-time work, and analyze their effects on health since working different number of hours could have different effects on health. Meanwhile, I also consider fixed effects as the data is panel, but this is irrelevant to discussion here.

The first stage regressions are both linear probability models. Since I have two endogenous variables, the instruments should provide independent sources of exogenous variation for both endogenous variables so that their effects can be identified. Hence, I consider the conditional F statistic (of Agrist and Pischke which is later improved by others; I do not present the results here) which suggests that the instruments are not weak.

The enclosed picture file presents the second stage results. The results show that the effect of part-time is much larger than the effect of full-time. But the referee points out a problem. Since both part-time and full time work are dummies, the larger are the first stage coefficients, and so the predicting power of the instruments, the smaller will be the IV coefficient (like in a Wald estimator). Therefore, it is almost mechanical to observe a larger estimated effect of working part-time on the health outcome because almost all instruments better predict the probability of working full-time than they do the probability of working part-time. In fact, a larger effect for part-time is observed for a couple of other health outcomes, supporting the referee's concern.

The referee's concern could be stated more simply as follows. Let us consider the simple IV estimator. It is given by cov(y,z)/cov(x,z) where y represents the dependent, x represents the single endogenous variable, z represents the single instrument, and there are no other exogenous covariates. According to the referee, if cov(x,z) in the sample is small, the IV estimate will be large.

I would like to ask two questions:

1. Given the critique, is the following then a lesson to be learned for the IV method in general? Suppose we have one endogenous variable and two instruments. Suppose we consider one instrument at a time: so no GIV but just IV estimation. Suppose both instruments are valid, equally significant, but that the first instrument has a larger effect on the endogenous variable than the second, in the first stage. If the referee is right, the first instrument will always result in a smaller IV estimate, and the second will result in a larger IV estimate, in the second stage. What do we conclude? If the effect of the instrument is large in the first stage, the IV estimate will be small in the second stage? But I do not recall myself reading about such a problem in any econometric textbook.

2. How could I proceed? To circumvent the critique, I should find an instrument for working part-time such that the effect of the instrument in the first stage is about the same size as that of the effect of the instrument for working full-time? It is probably not possible to find such an instrument. Should I discard the model all together? Or is there by chance an alternative econometric model I could turn to?

I do not fully understand your reviewer's point, but here are some thoughts...

1. If the effect of $x$ on $y$ is heterogeneous, then the two instruments identify different effects. Specifically, each instrument serves to identify a local average treatment effect for compliers (i.e. those induced to change $x$ by the instrument). In your case, the instrument identifies the local average treatment effect (LATE) for people induced to retire by the age change. The subpopulation that changes $x$ at 62 is different from the population that changes at 65.

2. Have you considered a regression discontinuity approach that focuses more narrowly on people around the key retirement ages? Changes in $y$ close to the retirement ages would be further support for your story if omitted variables $z$ that also cause $y$ don't change around the time of retirement.

Another question is what does the reduced form regression say? Does the instrument that is suppose to affect transitioning from part-time to retirement appear to affect $y$ less than the instrument supposedly affecting the transition from full-time to retirement?

Perhaps you need to restructure your argument to focus on (1) what you are trying to identify and (2) why your approach identifies it. This stuff about big versus small instruments seems tangential to me.

• 1. Can we take the significant different effects for the two endogenous variables in the 2nd stage as a formal indication that the instruments identify their effects? – Snoopy Jul 6 '17 at 12:44
• 2. I present graphs that show that y changes in a discontinuous manner at retirement ages. I did not consider RD design as I have 2 endogenous variables and six IVs. I thought the suitable model is the IV model. – Snoopy Jul 6 '17 at 12:44
• 3. 6 IVs affect both endogenous variables but to different degrees and in different directions. Hence, I also do not expect certain instruments to affect y in a certain direction, but they affect in the same direction but to different degrees. So since this could raise a concern, I conduct the Sanderson-Windmeijer test which suggests that the instruments provide independent sources of exogenous variation. I very much appreciate the reply, thanks. I would appreciate further comments. – Snoopy Jul 6 '17 at 12:49