# Bias of more than one endogenous variables

I may have a model with omitted variables that are correlated with my predictor variables. If I have in my model, let's say, two endogeneous variables X1, X2, but I am interested in obtaining an unbiased coefficient of just one variable X1, do I have to find instrument for both variables X1, X2? Or is it sufficient to instrument just for X1 and leave X2 alone?

X2 is just a minor control variable that I don't have an instrument for. However, my main interest is the coefficient on X1, for which I have a good instrument for. Is this okay to proceed?

In general, both. When you have an endogenous variable in a regression, all of the coefficient estimators are biased and inconsistent, not just the coefficient estimator for the coefficient on the endogenous variable. The $(X'X)^{-1}$ matrix spreads the bias around to the other coefficients as well. This also hints at the very special circumstances under which the bias is not spread around. If, for example, the endogenous variable is uncorrelated with all the other $X$ variables in the model, then the bias is not spread from the coefficient estimator for the endogenous variable to the coefficient estimators for the other variables.

EDIT: To respond to comment

The questioner does not find the answer above convincing. I'm not sure whether to respond by pushing algebra or by producing a baby monte carlo. I chose the latter. The monte carlo is written in R and appears below. It's a baby monte carlo because it only has one replication. On the other hand, it has 10,000 observations, so the law of large numbers has certainly kicked in. (Notice what the questioner calls X1 and X2, I call $X_2$ and $X_3$ in the monte carlo). The equation and true values for the parameters are: \begin{align} Y &= \beta_1 + \beta_2X_2 + \beta_3X_3 + \epsilon\\ \beta_1 &=0,\;\beta_2=1,\;\beta_3=1 \end{align}

Two things are varied in the monte carlo, the correlation between $X_2$ and $X_3$ and the correlation between $X_3$ and $W_2$, the instrument for $X_2$. The table below reports all the combinations of these correlations investigated in the monte carlo below. The final three columns of the table report the coefficient estimates for $\beta_2$ and $\beta_3$ using OLS, using two-stage least squares but only instrumenting for $X_2$, and using TSLS but instrumenting for both $X_2$ and $X_3$. Asterisked coefficients are coefficients which were reported as significantly different from their true values at 5% significance.

For example, the first row reports on a treatment in which the correlation among the $X$s was 0.8 and the correlation between the instrument for $X_2$ (called $W_2$) and $X_3$ is 0.4. In this case, OLS gave estimates for the betas which were biased upward, estimating them to be 1.22 instead of 1. These differences were significant. The one-instrument TSLS results had $\beta_2$ estimated at 0.23 and $\beta_3$ estimated at 2.05, both significantly different from their true values of 1. TSLS with both $X$s instrumented gave coefficients insignificantly different from their true values of 1.

\begin{array}{rrrrrr} \text{Tmt #} & cor(X_2,X_3) & cor(X_3,W_2) & \text{OLS} & \text{2SLS-1} & \text{2SLS-2}\\ \hline\\ 1 & 0.8 & 0.4 & 1.22^*,1.22^* & 0.23^*,2.05^* & 0.97,1.01\\ 2 & 0.0 & 0.4 & 1.45^*,1.45^* & 0.52^*,1.44^* & 0.97,0.99\\ 3 & 0.0 & 0.0 & 1.45^*,1.45^* & 0.98,1.44^* & 0.97,0.99\\ 4 & 0.8 & 0.0 & 1.22^*,1.22^* & 0.97,1.43^*& 0.97,1.00\\ \end{array}

What is shows is that instrumenting only for $X_2$ fails to correct the bias in the coefficient estimator for $X_2$'s coefficient. This strategy of instrumenting only for $X_2$ only corrects the bias if, in addition, $W_2$ and $X_3$ are uncorrelated---generally a strange and unlikely thing to have happen.

# This program written in response to a Cross Validated question
# http://stats.stackexchange.com/questions/96912/bias-of-more-than-one-endogenous-variables/96915#96915

# The program is a toy monte carlo.
# It generates two variables, X2 and X3, along with an error term ep.  The
# three of these are mutually correlated, so that both X1 and X2 are going
# to be endogenous in this equation:
#
# Y = b1 + b2*X2 + b3*X3 + ep
#
# Y is generated from that equation.  Then, an instrument for X2, called
# W2, is generated.  The equation above is estimated by 2SLS.
#
# The purpose of the monte carlo is to demonstrate that it is not enough
# to instrument for only one of the endogenous Xs in an equation, even if
# you only care about the coefficient on that variable.  That is, if
# we only instrument for X2 in the above equation, the IV estimator of
# its coefficient is _still_ inconsistent.

library(sem)

set.seed(12344321)

# Set the various basic parameters of the model.
# I am leaving the means of the Xs and Y at zero.  It is easy,
# though pointless, to change this.
var_X2 <- 1
var_X3 <- 1
var_W2 <- 1
var_W3 <- 1
var_ep <- 1
b1 <- 0
b2 <- 1
b3 <- 1

# Create large sample of normal errors
e1 <- rnorm(10000)
e2 <- rnorm(10000)
e3 <- rnorm(10000)
e4 <- rnorm(10000)
e5 <- rnorm(10000)

# Verify that they are uncorrelated and N(0,1)
cov(data.frame(e1,e2,e3,e4,e5))

# To create X2, X3, ep, and W2 I will use the following system
# of equations:
#
#  X2 = (e1 + g22*e2 + g2e*e3) * sqrt(var_X2)/sqrt(1+g22^2+g2e^2)
#  X3 = (e2 + g32*e1 + g3e*e3) * sqrt(var_X3)/sqrt(1+g32^2+g3e^2)
#  ep = sqrt(var_ep)*e3
#
# The "good" variation in X2 comes from e1 and e2.  The "bad," ie
# endogeneity-causing variation, in X2 comes from e3.  Same for
# X3.  The correlation between X2 and X3 comes from g2e and g3e.
# So, to make the endogeneity worse, increase the absolute value
# of g2e and g3e.
#
# An instrument, W2, for X2 should have none of the bad e3 variation
# in it and should have some of the good e1 and e2 variation in it.
# Same for W3.
#
# W2 = (e4 + h21*e1 + h22*e2) * sqrt(var_W2)/sqrt(1+h21^2+h22^2)
# W3 = (e5 + h31*e1 + h32*e2) * sqrt(var_W3)/sqrt(1+h31^2+h32^2)
#
# The covariance between W2 and X2 (and X3 for that matter) comes from
# h21 and h22.  Thus these parameters controll how good an instrument
# W2 is for X2.  As long as all these parameters are positive (as they
# are below in my example), increasing h22, h23 make W2 a better intrument
# for X2.

# Setting params which controll endogenity and instrument quality:
g22 <- 0.5
g2e <- 0.5
g32 <- 0.5
g3e <- 0.5
h21 <- 0.75
h22 <- 0.25
h31 <- 0.25
h32 <- 0.75

# Generating data:
X2 <- (e1 + g22*e2 + g2e*e3) * sqrt(var_X2)/sqrt(1+g22^2+g2e^2)
X3 <- (e2 + g32*e1 + g3e*e3) * sqrt(var_X3)/sqrt(1+g32^2+g3e^2)
ep <- sqrt(var_ep)*e3
W2 <- (e4 + h21*e1 + h22*e2) * sqrt(var_W2)/sqrt(1+h21^2+h22^2)
W3 <- (e5 + h31*e1 + h32*e2) * sqrt(var_W3)/sqrt(1+h31^2+h32^2)
Y  <- b1 + b2*X2 + b3*X3 + ep

# OK, now the various correlations among these variables should be:

# cor(X2,X3) = (g22 + g32 + g2e*g3e)/sqrt((1+g22^2+g2e^2)*(1+g32^2+g3e^2))
(g22 + g32 + g2e*g3e)/sqrt((1+g22^2+g2e^2)*(1+g32^2+g3e^2))
cov(X2,X3)

# cor(X2,ep) = g2e*var_ep/sqrt((1+g22^2+g2e^2)*(var_ep))
g2e*var_ep/sqrt((1+g22^2+g2e^2)*(var_ep))
cov(X2,ep)

# cor(X3,ep) = g3e*var_ep/sqrt((1+g32^2+g3e^2)*(var_ep))
g3e*var_ep/sqrt((1+g32^2+g3e^2)*(var_ep))
cov(X3,ep)

# cor(X2,W2) = (h21+h22*g22) / sqrt((1+g22^2+g2e^2)*(1+h21^2+h22^2))
(h21+h22*g22) / sqrt((1+g22^2+g2e^2)*(1+h21^2+h22^2))
cov(X2,W2)

# A look at the whole variance matrix & correlation matrix:
var(data.frame(Y,X2,X3,W2,W3,ep))
cor(data.frame(Y,X2,X3,W2,W3,ep))

# OLS should be badly biased:
summary(lm(Y~X2+X3))

# TSLS using both instruments should be fine:
summary(tsls(Y~X2+X3,~W2+W3))

# TSLS using only W2 should not be fine:
summary(tsls(Y~X2+X3,~W2+X3))

# How about if we get rid of cross-correlation among the X:
g22 <- -g2e*g3e/2
g32 <- -g2e*g3e/2

# Maintain the correlation between X3 and W2, though:
(h22+h21*g32) / sqrt((1+g32^2+g3e^2)*(1+h21^2+h22^2))
h22 <- 0.75
(h22+h21*g32) / sqrt((1+g32^2+g3e^2)*(1+h21^2+h22^2))

# Generating data:
X2 <- (e1 + g22*e2 + g2e*e3) * sqrt(var_X2)/sqrt(1+g22^2+g2e^2)
X3 <- (e2 + g32*e1 + g3e*e3) * sqrt(var_X3)/sqrt(1+g32^2+g3e^2)
ep <- sqrt(var_ep)*e3
W2 <- (e4 + h21*e1 + h22*e2) * sqrt(var_W2)/sqrt(1+h21^2+h22^2)
W3 <- (e5 + h31*e1 + h32*e2) * sqrt(var_W3)/sqrt(1+h31^2+h32^2)
Y  <- b1 + b2*X2 + b3*X3 + ep

# A look at the whole variance matrix & correlation matrix:
var(data.frame(Y,X2,X3,W2,W3,ep))
cor(data.frame(Y,X2,X3,W2,W3,ep))

# OLS should be badly biased:
summary(lm(Y~X2+X3))

# TSLS using both instruments should be fine:
summary(tsls(Y~X2+X3,~W2+W3))

# TSLS using only W2 should not be fine:
summary(tsls(Y~X2+X3,~W2+X3))

# How about if we get rid of correlation between W2 and X3 also.
# cor(W2,X3) = (h22+h21*g32)/sqrt((1+g32^2+g3e^2)*(1+h21^2+h22^2))
h22 <- -h21*g32

# Generating data:
X2 <- (e1 + g22*e2 + g2e*e3) * sqrt(var_X2)/sqrt(1+g22^2+g2e^2)
X3 <- (e2 + g32*e1 + g3e*e3) * sqrt(var_X3)/sqrt(1+g32^2+g3e^2)
ep <- sqrt(var_ep)*e3
W2 <- (e4 + h21*e1 + h22*e2) * sqrt(var_W2)/sqrt(1+h21^2+h22^2)
W3 <- (e5 + h31*e1 + h32*e2) * sqrt(var_W3)/sqrt(1+h31^2+h32^2)
Y  <- b1 + b2*X2 + b3*X3 + ep

# A look at the whole variance matrix & correlation matrix:
var(data.frame(Y,X2,X3,W2,W3,ep))
cor(data.frame(Y,X2,X3,W2,W3,ep))

# OLS should be badly biased:
summary(lm(Y~X2+X3))

# TSLS using both instruments should be fine:
summary(tsls(Y~X2+X3,~W2+W3))

# TSLS using only W2 should not be fine:
summary(tsls(Y~X2+X3,~W2+X3))

# How about if we get rid of correlation between W2 and X3 but
# go back to having X2 and X3 correlated.
# cor(W2,X3) = (h22+h21*g32)/sqrt((1+g32^2+g3e^2)*(1+h21^2+h22^2))
g22 <- 0.5
g32 <- 0.5
h22 <- -h21*g32

# Maintain the correlation between X2 and W2, though:
(h21+h22*g22) / sqrt((1+g22^2+g2e^2)*(1+h21^2+h22^2))
h21 <- 0.75
(h21+h22*g22) / sqrt((1+g22^2+g2e^2)*(1+h21^2+h22^2))

# Check cor(W3,X3) reasonable
# cor(W3,X3) = (h31*g32+h32)/sqrt((1+h31^2+h32^2)*(1+g32^2+g3e^2))
(h31*g32+h32)/sqrt((1+h31^2+h32^2)*(1+g32^2+g3e^2))

# Generating data:
X2 <- (e1 + g22*e2 + g2e*e3) * sqrt(var_X2)/sqrt(1+g22^2+g2e^2)
X3 <- (e2 + g32*e1 + g3e*e3) * sqrt(var_X3)/sqrt(1+g32^2+g3e^2)
ep <- sqrt(var_ep)*e3
W2 <- (e4 + h21*e1 + h22*e2) * sqrt(var_W2)/sqrt(1+h21^2+h22^2)
W3 <- (e5 + h31*e1 + h32*e2) * sqrt(var_W3)/sqrt(1+h31^2+h32^2)
Y  <- b1 + b2*X2 + b3*X3 + ep

# A look at the whole variance matrix & correlation matrix:
var(data.frame(Y,X2,X3,W2,W3,ep))
cor(data.frame(Y,X2,X3,W2,W3,ep))

# OLS should be badly biased:
summary(lm(Y~X2+X3))

# TSLS using both instruments should be fine:
summary(tsls(Y~X2+X3,~W2+W3))

# TSLS using only W2 should not be fine:
summary(tsls(Y~X2+X3,~W2+X3))

• thanks, but if I instrument for X1, does this not already correct the bias? Since I am only interested in the coefficient on X1 and don't care about X2, which is a minor control variable in my case.... – CodeTrek May 8 '14 at 12:40
• No. The endogeneity bias coming from $X_2$ will be spread to $X_1$, unless some special condition holds. – Bill May 8 '14 at 13:02
• It is clear that it is spread to X1, but that's why I have an instrument to purge X1 from endogeneity. – CodeTrek May 8 '14 at 13:08
• OK, added a monte carlo to make the point more clear. – Bill May 9 '14 at 14:56
• "This also hints at the very special circumstances under which the bias is not spread around. If, for example, the endogenous variable is uncorrelated with all the other X variables in the model, then the bias is not spread from the coefficient estimator for the endogenous variable to the coefficient estimators for the other variables." Would it be possible to provide a source (textbook, article) discussing this issue? Ivan – Ivan Jun 5 '19 at 8:34