What does "validity of an instrument" mean exactly?

In my econometrics course we have just defined instrument validity as $E[Z|u]=0$, where $Z$ is the instrumental variable and $u$ is the error term of a univariate regression model. Then, we also talked about strength of an instrument, but I am pretty sure I have understood correctly that it is a different requirement than validity.

In applications, I often find the definition of validity as ${\rm corr}(Z,X) \neq 0$, where $Z$ is the instrument and $X$ is the endogenous explanatory variable, plus the requirement that $E[Z|u]=0$ (as above), which is usually defined as exclusion restriction.

I am a bit confused and it is not so easy to find the kind of a primer on IV approaches I need. Is anyone able to unravel these issues?

  • $\begingroup$ This question might be better suited for the Economics website for Stack Exchange. $\endgroup$ – Mike Hunter May 25 '16 at 20:58
  • 4
    $\begingroup$ @DJohnson, I think this could be on either. Understanding instrumental variables is a statistical topic. When a question could be on topic on more than one site, I typically defer to the OP's choice. $\endgroup$ – gung - Reinstate Monica May 25 '16 at 21:01
  • $\begingroup$ @DJohnson I think it is appropriate for CV: IV estimation is certainly not restricted to economics/econometrics in application (although the technique originated within the econometrics discipline). Epidemiology papers and textbooks, such as the one I cited in my answer (and I can think of others off of the top of my head), address IV estimation and IV variable identification methods. $\endgroup$ – Alexis May 25 '16 at 21:30

Requirements for Z to be a valid instrument for X are:

  • Relevance = Z needs to highly correlated with X
  • Exogenous = Z is correlated with Y solely through its correlation with X; so Z is uncorrelated with the error in the outcome equation

The main idea behind IV is that when Z changes, it should also alter X, but not the troublesome part of X that is correlated with the error. To get the effect of X on Y we are only using part of the variation in X, the part that's driven by variation in Z.

| cite | improve this answer | |
  • $\begingroup$ This is a good laymen's terms explanation of what IV estimation does: "but not the troublesome part of X that is correlated with the error." There's a funny little (OK, not so little, being 30 mins long) video by Antonakis on YouTube about endogeneity, where it depicted as a spotty miasma-being for added emphasis on its troublesome nature! $\endgroup$ – Marquis de Carabas May 25 '16 at 21:29
  • 1
    $\begingroup$ highly is too strong a word. $Z$ needs to be correlated with $X$, with the caveat that if the relationship is too weak, you'll likely run into the well known problems associated with weak instruments. $\endgroup$ – Matthew Gunn May 25 '16 at 21:30
  • $\begingroup$ @MatthewGunn I didn't specify an exact threshold, so highly is the eye of the beholder. Staiger and Stock's '97 Econometrica paper argues that the finite sample bias (towards the plim of OLS) is proportional to the first-stage F-statistic, so higher is always better in my mind. $\endgroup$ – Dimitriy V. Masterov May 25 '16 at 22:04
  • $\begingroup$ Agreed that "highly" is one of those terms open to interpretation and that higher correlation is better. To quote the Rolling Stones though, "you can't always get what you want, but if you try sometime, you just might find, you get what you need." :P I'd personally write "$Z$ is sufficiently correlated with $X$: estimates based on weak instruments may exhibit significant finite sample bias." $\endgroup$ – Matthew Gunn May 25 '16 at 22:22
  • 1
    $\begingroup$ @user001 No, this assumption in unverifiable. Insignificance in that specifiaction tells you very little. $\endgroup$ – Dimitriy V. Masterov Dec 26 '18 at 21:55

Following Hernán and Robins' Causal Inference, Chapter 16: Instrumental variable estimation, instrumental variables have four assumptions/requirements:

  1. $Z$ must be associated with $X$.

  2. $Z$ must causally affect $Y$ only through $X$

  3. There must not be any prior causes of both $Y$ and $Z$.

  4. The effect of $X$ on $Y$ must be homogeneous. This assumption/requirement has two forms, weak and strong:

    • Weak homogeneity of the effect of $X$ on $Y$: The effect of $X$ on $Y$ does not vary by the levels of $Z$ (i.e. $Z$ cannot modify the effect of $X$ on $Y$).
    • Strong homogeneity of the effect of $X$ on $Y$: The effect of $X$ on $Y$ is constant across all individuals (or whatever your unit of analysis is).

Instruments that do not meet these assumptions are generally invalid. (2) and (3) are generally difficult to provide strong evidence for (hence assumptions).

The strong version of condition (4) may be a very unreasonable assumption to make depending on the nature of the phenomena being studied (e.g. the effects of drugs on individuals' health generally varies from individual to individual). The weak version of condition (4) may require the use of atypical IV estimators, depending on the circumstance.

The weakness of the effect of $Z$ on $X$ does not really have a formal definition. Certainly IV estimation produces biased results when the effect of $Z$ on $X$ is small relative to the effect of $U$ (unmeasured confounder) on $X$, but there's no hard and fast point, and the bias depends on sample size. Hernán and Robins are (respectfully and constructively) critical of the utility of IV regression relative to estimates based on formal causal reasoning of their approach (that is, the formal causal reasoning approach of the counterfactual causality folks like Pearl, etc.).

Hernán, M. A. and Robins, J. M. (2017). Causal Inference. Chapman & Hall/CRC.

| cite | improve this answer | |
  • $\begingroup$ How are you able to reference and quote from this book? According to Amazon, it's not being published until December of this year. $\endgroup$ – Mike Hunter May 25 '16 at 21:31
  • 1
    $\begingroup$ @DJohnson Follow my link (they make the pre-press pdfs available). ;) Also, I took their class 15 years ago and they were dissecting it even then. $\endgroup$ – Alexis May 25 '16 at 21:33
  • $\begingroup$ @Alexis What is the intuition on why you need homogeneity? $\endgroup$ – Dimitriy V. Masterov May 25 '16 at 22:20
  • $\begingroup$ @DimitriyV.Masterov It's a sophisticated argument (see Technical point 16.3 in Hernán and Robins), but amounts to the insufficiency of assumptions/requirements 1–3 to fully identify the average causal effect of $X$ on $Y$. $\endgroup$ – Alexis May 25 '16 at 22:51

Both assumptions can be seen by looking at the system of equations:

\begin{align} x=&\gamma_1+\gamma_2 z+\epsilon\\ y=&\beta_1+\beta_2 x+\gamma_3 z+u \end{align}

  • The strength of the instrument relates to the coefficient $\gamma_2\neq 0$ and to the $R^2$ of this equation (both should be high enough)

  • The validity relates to the assumption that $\gamma_3=0$, i.e. $z$ has no direct effect on $y$.

Note that we cannot test $\gamma_3=0$, only assume it, which explains why it is called an identifying (=untestable) assumption.

| cite | improve this answer | |
  • 1
    $\begingroup$ The problem with respect to the strength of the instrument is that "high enough" does not really have a formal definition. $\endgroup$ – Alexis May 26 '16 at 2:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.