How does IV 2SLS obtain a causal coefficient?

Question

Despite reading and conducting several practical examples with IV 2SLS, I am still uncertain how, specifically and mathematically, 2SLS is able to obtain a causal coefficient, β, of an assumed endogenous variable, X, on an outcome, Y.

From what I gather, 2SLS follows this logic:

1. First stage: We regress the endogenous variable, X, on all exogenous variables including the instrument(-s). We then store the predicted value of X.

2. Second stage: In the second stage regression, the predicted value of X now replaces the endogenous variable, consequently β now represents an "isolated" causal coefficient for X on Y.

At a conceptual level, I understand that the first stage somehow removes the correlation between the X variable and the error term, ϵ. So, when we in the second stage replace X with the predicted value of X we obtain a causal coefficient, β, for the effect of X on Y. However, I am unsure about the mathematics regarding the "isolation" of the causal effect of X on Y. Thus, the main question is what is the specific mathematical operation that makes β a causal coefficient for the effect of X on Y in the second stage regression?

Another post (What is an instrumental variable?) vividly describes how 2SLS can single out the explained and unexplained variation of an endogenous variable by the two-stage procedure. However, the example is based on a first stage that regresses the endogenous variable on the instrument, thereafter you plug the predicted value of X into the second stage regression. While illustrating, I am unsure how this translates to a more conventional example where you use 2SLS with an endogenous variable, multiple explanatory variables and one instrument.

Answer 1

Start from the structural model, $$y_i = \alpha + \beta X_i + \epsilon_i$$ where the explanatory variable of interest $X_i$ has a correlation with the error term, $Cov(X_i,\epsilon_i)\neq 0$. In this case, you know that you won't recover an unbiased estimate such that $\widehat{\beta}\rightarrow \beta$. Now assume that you have an instrument $Z_i$ which is such that $Cov(X_i,Z_i)\neq0$ and $Cov(Z_i,\epsilon_i)=0$. These are the assumptions on instrument relevance and the exclusion restriction.

Take the covariance of each sides of the above equation with respect to $Z_i$, and you get $$ \begin{align} Cov(y_i,Z_i) &= \beta Cov(X_i,Z_i) + Cov(\epsilon_i, Z_i)\\[0.5em] \beta &= \frac{Cov(y_i,Z_i)}{Cov(X_i,Z_i)} \end{align} $$ which uses the fact that the covariance between a random variable and a constant is zero, as well as the previous exclusion restriction. In fact, we just derived the expression of the IV estimator. The population coefficient $\beta$ can be recovered by dividing the "reduced form" (regression of $y_i$ on $Z_i$) coefficient by the first stage (regression of $X_i$ on $Z_i$) coefficient.

How does this relate to my answer in the other post? The denominator of the above fraction can be obtained by regressing, $$ X_i = \delta + \pi Z_i + \eta_i $$

Now you see that we have an expression for $X_i$ as a linear function of the instrument. If you plug this into the very first equation, you get the so-called reduced form equation: $$ \begin{align} y_i &= \alpha + \beta X_i + \epsilon_i \\ &= \alpha + \beta (\delta + \pi Z_i + \eta_i) + \epsilon_i \\ &= (\alpha + \beta\delta) + \beta \pi Z_i + \beta\eta_i\epsilon_i \end{align} $$

So the ratio of the reduced form coefficient on $Z_i$ over the first stage coefficient is indeed $$ \begin{align} \beta &= \frac{Cov(y_i,Z_i)}{Cov(X_i,Z_i)} \\ &= \frac{\beta\pi}{\pi}\\ &= \beta \end{align} $$ the causal effect of interest. This is the maths behind it. I hope that this together with the other answer gives you a better intuition on how an instrumental variable can be used to extract "exogenous" variation (under the stated assumptions) from the original $X_i$ to identify the parameter of interest.

Answer 2

Justification for instrumental variables are in nature theoretical narrative compared to fully statistical. They are also called quasi-experimental.

In a perfect world we would always have well implemented randomized trials but in the reality this is rarely the case, and thus economists have started to exploit these quasi-random variations in population. Well implemented randomized trials by definition do not have correlated X and error term.

Examples are always enlightening. Let me introduce to you my favourite one:

Length of breastfeeding for newborns is assumed to be big factor on their growth. Observational studies are not reliable because not all endogenous variables can be controlled (omitted variable bias). What kind of random variation was exploited in this case?

Researchers realised that during the weekends hospitals had less people working and thus nurses had less time to educate mothers on the benefits of the breastfeeding. Starting childbirth is in this case assumed to be random and you go to hospital independently of the day of the week. Instrumental variable is the days when mother's didn't receive same depth of instruction (weekends). This would remove the correlation between X and error term.

I hope you can see where the quasi-experimental setting comes from. In the absence of randomized trial we hope this observable randomness is good enough to be best possible replacement for it. There is still a lot to complain about the previous IV e.g. is the starting of childbirth actually random?; Is the effect of instructions big enough to cause differences in breastfeeding?

Intuition in 3 steps:

• Instrument affects who is assigned to the treatment (X)
• Instruments is unrelated to other factors that affect the outcome.
• If and only if you know that instrument has no direct impact on the outcome,it can only affect outcome by it's impact on treatment (X)

When you see papers with instrumental variables they go in lengthy detail on this narrative why it's actually random variation with supporting evidence from data. E.g. mother from weekdays breastfeed x times longer and y times as frequently.

I feel that if people would understand IV just as replacement for randomized trial they wouldn't get boggled down with mathematics.