0
$\begingroup$

I have two questions.

I am conducting the Hausman test to check the endogeneity of a variable. If the Hausman test fails to reject the null hypothesis, there is no difference between OLS (my reference model) and 2SLS estimates. The OLS estimates are consistent.

Someone argued that even if we do not reject the null (and the instruments are valid and not weak), we cannot necessarily rule out endogeneity. In other words, this person claimed that, if we have reasons to believe that the variable is endogenous, we have to use the 2SLS, even if there is no evidence of endogeneity (based on the results from the Hausman test).

1) Is that correct?

I am thinking that an important question is about the magnitude of the endogeneity. For example, if you have a sample of 100k people that is representative of the population of interest, and the variable you are interested in is endogenous for, say, 100 people (these 100 people have a common, observable, and known characteristic, and they are able to manipulate the value of that variable), the OLS might still give estimates statistically indistinguishable from the 2SLS (i.e. the OLS is consistent). There is endogeneity, but its magnitude is so small that the OLS gives the same estimates as the 2SLS.

2) Should you still use the 2SLS based on logical grounds (i.e. you know there is endogeneity caused by a known characteristic)?

[there is this one similar question, but it is never addressed the case when the instrument is not weak and it is; plus, it is not asked whether it is theoretically possible to have endogeneity, but so small that the OLS could appear to be just fine]

$\endgroup$
1
$\begingroup$

Let me begin my answer to question 1) by noting that the Hausman test, as with any other statistical test, relies on certain assumptions being true/good approximations. For example, in IV settings, an assumption that can really go awry is if you have a "weak" instrument in the sense that your instrument does not do a very good job of explaining $X$ (and in particular, is barely statistically distinguishable from having no effect on $X$ at all). You should always be wary that this can cause your test to incorrectly reject a null hypothesis too. Assuming that you are sure about your assumptions, your intuition that the "magnitude of endogeneity" complicates the analysis is correct, but this concern potentially cuts deeper than you realize. Recall that if a Hausman test fails to reject that the 2SLS slope is different from the OLS slope implies that the data does not contain sufficient evidence to reject the null hypothesis that the two slopes are identical. In general, there are two reasons why you might get a null result in hypothesis testing: the null hypothesis is true and you correctly failed to reject it, or the null hypothesis is false, but you incorrectly failed to reject it.

The main concern, as you seem to have correctly identified, is that you might fail to reject the null for the second reason. In your example, the reason why you fail to reject the null hypothesis is that the null of no endogeneity is technically false, but the magnitude of violation is so small that for all intents and purposes, it would be unreasonable to expect a test to pick up on this difference. In your example, this does not seem so concerning - you failed to correctly reject a null hypothesis that was technically wrong, but for all practical intents and purposes, the null was not even that strongly violated, so you would hardly even make much of an error proceeding as if it was true. However, there are other examples where you fail to correct reject a false null hypothesis for much more concerning reasons. To take an extreme example, suppose that OLS a very precise point estimate of $\beta_1 = 1$ with a standard error of $0.01$. On the other hand, suppose that your IV estimate gave you a point estimate of $\beta = -1$, but with a huge standard error of $10$. In this case, the 95% confidence interval for IV could be anywhere between -21 and 19, which would cause a statistical test of whether the two slopes are identical to fail to reject. But notice that in this extreme example, this test is really not telling us much at all. This conclusion that we cannot reject the null hypothesis would occur in cases as polar as OLS being badly biased downwards (e.g. $\beta_1 = 10$ in reality) or OLS giving the wrong sign (e.g. $\beta_1 = -5$ in reality).

In general, the concerns I raised above can be mitigated by being more transparent about what is going on. For example, rather than mechanically relying on the results of the Hausman test, it is possible to get an intuitive feel for what the Hausman test is actually picking up on by separately looking at point estimates and standard errors for IV and OLS respectively. Do the point estimates look similar? How wide do the confidence intervals look? However, even if you do that, you should be aware that the Hausman test is asking the question of whether your particular 2SLS estimates are different from your OLS estimates. If your 2SLS estimates themselves suffer from endogeneity problems, then you have not ruled out endogeneity and have just shown that the endogeneity plaguing 2SLS seems to be similar to the endogeneity affecting OLS.

I think giving an answer to 2) that works in every scenario is difficult to provide. Causal inference is difficult and highly contingent on domain knowledge about the idiosyncrasies of your setting, but some general guidelines might be useful.

  1. To cite an overused quote, "All models are wrong, but some are useful". For OLS to have a causal interpretation, your model of the world has to be something of the form "there is no systematic relationship between the assignment of my treatment $X$ and other factors that might affect my outcome $Y$". If you truly believe this is a good model of the world (or at least a close enough model), then you can use OLS (perhaps with some caution). I should point out though, that at least in social sciences, there are few reasons to believe that such an assumption is a good one unless $X$ is experimentally manipulated. To the extent that people have an ability to control $X$, you can typically expect them to do their best to satisfy whatever objectives they might have, so when $Y$ is an outcome people care about, you expect people to choose $X$ based on their anticipated outcome $Y$, which will cause the "other factors" to systematically be related to $X$.
  2. If you had known and measurable characteristics that are causing endogeneity, you should do your best to control for these influences.
  3. If you have unmeasurable characteristics causing endogeneity, then you know OLS will be biased. An instrument that truly satisfied all of the usual IV assumptions and in addition does not suffer from statistical deficiencies such as a "weak instrument" problem will typically be a more reliable estimate than OLS, but when you think your IV is not literally satisfying its assumptions but merely "less wrong" than OLS, the cure may be worse than the disease. For example, even if you do not have a weak instrument problem in the statistical sense, in general, it will be the case that the weaker the instrument, the more that endogeneity problems of the instrument itself will be "magnified". This can be seen in the single instrument case by noting that $\hat\beta_{IV} = Cov(Z,Y) / Cov(Z,X)$. Since $Cov(Z,X)$ is on the denominator, any endogeneity problem will get divided by a smaller number, the weaker an instrument is. Thus, even if $Cov(Z,X)$ is bounded away from 0, it can still cause problems when it is small.
  4. Something to be aware of is that a lot of the traditional textbook theory of IV implicitly assumes homogeneous treatment effects. Under the presence of heterogeneous treatment effects, the analysis is even more tricky. Good resources to look at to at least be aware of these issues involved are Imbens and Angrist (1994) and Imbens, Angrist, and Graddy (2000).
  5. As a bottom line, it is your job to do your homework and convince your audience that what you are estimating is reasonable for your particular situation. If justifying OLS, you should be prepared to argue why $X$ is not correlated with other factors affecting $Y$. In particular, you should be prepared for critics to think up myriad counterexamples of such confounding variables. To the extent that you can control for those, you should. If you wanted to use an instrument, you should be prepared to justify why it makes sense to use it. Why can we treat the instrument $Z$ as "as good as randomly assigned" when we could not assume that of $X$? Does the instrument suffer from obvious statistical deficiency tat makes inference with it invalid?
$\endgroup$

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .