Despite reading and conducting several practical examples with IV 2SLS, I am still uncertain how, specifically and mathematically, 2SLS is able to obtain a causal coefficient, β, of an assumed endogenous variable, X, on an outcome, Y.

From what I gather, 2SLS follows this logic:

  1. First stage: We regress the endogenous variable, X, on all exogenous variables including the instrument(-s). We then store the predicted value of X.

  2. Second stage: In the second stage regression, the predicted value of X now replaces the endogenous variable, consequently β now represents an "isolated" causal coefficient for X on Y.

At a conceptual level, I understand that the first stage somehow removes the correlation between the X variable and the error term, ϵ. So, when we in the second stage replace X with the predicted value of X we obtain a causal coefficient, β, for the effect of X on Y. However, I am unsure about the mathematics regarding the "isolation" of the causal effect of X on Y. Thus, the main question is what is the specific mathematical operation that makes β a causal coefficient for the effect of X on Y in the second stage regression?

Another post (What is an instrumental variable?) vividly describes how 2SLS can single out the explained and unexplained variation of an endogenous variable by the two-stage procedure. However, the example is based on a first stage that regresses the endogenous variable on the instrument, thereafter you plug the predicted value of X into the second stage regression. While illustrating, I am unsure how this translates to a more conventional example where you use 2SLS with an endogenous variable, multiple explanatory variables and one instrument.


Start from the structural model, $$y_i = \alpha + \beta X_i + \epsilon_i$$ where the explanatory variable of interest $X_i$ has a correlation with the error term, $Cov(X_i,\epsilon_i)\neq 0$. In this case, you know that you won't recover an unbiased estimate such that $\widehat{\beta}\rightarrow \beta$. Now assume that you have an instrument $Z_i$ which is such that $Cov(X_i,Z_i)\neq0$ and $Cov(Z_i,\epsilon_i)=0$. These are the assumptions on instrument relevance and the exclusion restriction.

Take the covariance of each sides of the above equation with respect to $Z_i$, and you get $$ \begin{align} Cov(y_i,Z_i) &= \beta Cov(X_i,Z_i) + Cov(\epsilon_i, Z_i)\\[0.5em] \beta &= \frac{Cov(y_i,Z_i)}{Cov(X_i,Z_i)} \end{align} $$ which uses the fact that the covariance between a random variable and a constant is zero, as well as the previous exclusion restriction. In fact, we just derived the expression of the IV estimator. The population coefficient $\beta$ can be recovered by dividing the "reduced form" (regression of $y_i$ on $Z_i$) coefficient by the first stage (regression of $X_i$ on $Z_i$) coefficient.

How does this relate to my answer in the other post? The denominator of the above fraction can be obtained by regressing, $$ X_i = \delta + \pi Z_i + \eta_i $$

Now you see that we have an expression for $X_i$ as a linear function of the instrument. If you plug this into the very first equation, you get the so-called reduced form equation: $$ \begin{align} y_i &= \alpha + \beta X_i + \epsilon_i \\ &= \alpha + \beta (\delta + \pi Z_i + \eta_i) + \epsilon_i \\ &= (\alpha + \beta\delta) + \beta \pi Z_i + \beta\eta_i\epsilon_i \end{align} $$

So the ratio of the reduced form coefficient on $Z_i$ over the first stage coefficient is indeed $$ \begin{align} \beta &= \frac{Cov(y_i,Z_i)}{Cov(X_i,Z_i)} \\ &= \frac{\beta\pi}{\pi}\\ &= \beta \end{align} $$ the causal effect of interest. This is the maths behind it. I hope that this together with the other answer gives you a better intuition on how an instrumental variable can be used to extract "exogenous" variation (under the stated assumptions) from the original $X_i$ to identify the parameter of interest.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your quick and excellent answer. This was very helpful! $\endgroup$ – Tarjei W. Havneraas May 28 '18 at 22:11
  • $\begingroup$ @Andy may I ask why you call that representation $y_i= \alpha + \beta X_i + \epsilon_i$ a "structural model"? $\endgroup$ – suckrates May 29 '18 at 7:45
  • 1
    $\begingroup$ @AlvaroFuentes this equation is called a "structural equation" or a "structural model" to emphasize that the equation is supposed to measure a causal relationship. Basically, a structural equation, at least in econometrics, is defined as "an equation derived from economic theory or from less formal economic reasoning". See e.g. Wooldridge (2016) Introductory Econometrics. Andy, I think I follow the full rationale of your answer. However, I am a bit confused about the new notations introduced in this equation: Xi=δ+πZi+ηi. Could you please elaborate? $\endgroup$ – Tarjei W. Havneraas May 29 '18 at 8:39
  • $\begingroup$ @TarjeiW.Havneraas I see. I was thinking about structural vs reduced-form, but the source you give clears it up. Thanks. $\endgroup$ – suckrates May 29 '18 at 8:47
  • 1
    $\begingroup$ @TarjeiW.Havneraas the regression $X_i = \delta +\pi Z_i + \eta_i$ is the so-called "first stage" regression. The change in notation is to show that it is a separate regression equation. In principle, you could first run this regression, obtain the fitted values $\widehat{X}_i$, and then obtain the 2sls estimate by regressing $y_i = \alpha + \beta \widehat{X}_i + \epsilon_i$ $\endgroup$ – Andy Jun 6 '18 at 21:27

Justification for instrumental variables are in nature theoretical narrative compared to fully statistical. They are also called quasi-experimental.

In a perfect world we would always have well implemented randomized trials but in the reality this is rarely the case, and thus economists have started to exploit these quasi-random variations in population. Well implemented randomized trials by definition do not have correlated X and error term.

Examples are always enlightening. Let me introduce to you my favourite one:

Length of breastfeeding for newborns is assumed to be big factor on their growth. Observational studies are not reliable because not all endogenous variables can be controlled (omitted variable bias). What kind of random variation was exploited in this case?

Researchers realised that during the weekends hospitals had less people working and thus nurses had less time to educate mothers on the benefits of the breastfeeding. Starting childbirth is in this case assumed to be random and you go to hospital independently of the day of the week. Instrumental variable is the days when mother's didn't receive same depth of instruction (weekends). This would remove the correlation between X and error term.

I hope you can see where the quasi-experimental setting comes from. In the absence of randomized trial we hope this observable randomness is good enough to be best possible replacement for it. There is still a lot to complain about the previous IV e.g. is the starting of childbirth actually random?; Is the effect of instructions big enough to cause differences in breastfeeding?

Intuition in 3 steps:

  • Instrument affects who is assigned to the treatment (X)
  • Instruments is unrelated to other factors that affect the outcome.
  • If and only if you know that instrument has no direct impact on the outcome,it can only affect outcome by it's impact on treatment (X)

When you see papers with instrumental variables they go in lengthy detail on this narrative why it's actually random variation with supporting evidence from data. E.g. mother from weekdays breastfeed x times longer and y times as frequently.

I feel that if people would understand IV just as replacement for randomized trial they wouldn't get boggled down with mathematics.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your reply and your illustrating example. I agree that its helpful to view IV as a replacement for RCT, and I have positive experience with presenting it as such to "outsiders". As I work on a project where we will use IV, I just really want to get all the mathematics right as well to have a solid understanding of each step involved in the estimation procedure. $\endgroup$ – Tarjei W. Havneraas May 29 '18 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.