# Estimating $b_1 x_1+b_2 x_2$ instead of $b_1 x_1+b_2 x_2+b_3x_3$

I have a theoretical economic model which is as follows,

$$y = a + b_1x_1 + b_2x_2 + b_3x_3 + u$$

So theory says that there are $x_1$, $x_2$ and $x_3$ factors to estimate $y$.

Now I have the real data and I need to estimate $b_1$, $b_2$, $b_3$. The problem is that the real data set contains only data for $x_1$ and $x_2$; there are no data for $x_3$. So the model I can fit actually is:

$$y = a + b_1x_1 + b_2x_2 + u$$

• Is it OK to estimate this model?
• Do I lose anything estimating it?
• If I do estimate $b_1$, $b_2$, then where does the $b_3x_3$ term go?
• Is it accounted for by error term $u$?

And we would like to assume that $x_3$ is not correlated with $x_1$ and $x_2$.

• Can you give details about your data set, I mean, your dependent variable $y$ and independent variables $x_1$ and $x_2$? – Vara May 11 '13 at 14:32
• Think of it as hypothethical example without specific data set... – renathy May 11 '13 at 15:05

The issue you need to worry about is called endogeneity. More specifically, it depends on whether $x_3$ is correlated in the population with $x_1$ or $x_2$. If it is, then the associated $b_j$s will be biased. That is because OLS regression methods force the residuals, $u_i$, to be uncorrelated with your covariates, $x_j$s. However, your residuals are composed of some irreducible randomness, $\varepsilon_i$, and the unobserved (but relevant) variable, $x_3$, which by stipulation is correlated with $x_1$ and / or $x_2$. On the other hand, if both $x_1$ and $x_2$ are uncorrelated with $x_3$ in the population, then their $b$s won't be biased by this (they may well be biased by something else, of course). One way econometricians try to deal with this issue is by using instrumental variables.

For the sake of greater clarity, I've written a quick simulation in R that demonstrates the sampling distribution of $b_2$ is unbiased / centered on the true value of $\beta_2$, when it is uncorrelated with $x_3$. In the second run, however, note that $x_3$ is uncorrelated with $x_1$, but not $x_2$. Not coincidentally, $b_1$ is unbiased, but $b_2$ is biased.

library(MASS)                          # you'll need this package below
N     = 100                            # this is how much data we'll use
beta0 = -71                            # these are the true values of the
beta1 = .84                            # parameters
beta2 = .64
beta3 = .34

############## uncorrelated version

b0VectU = vector(length=10000)         # these will store the parameter
b1VectU = vector(length=10000)         # estimates
b2VectU = vector(length=10000)
set.seed(7508)                         # this makes the simulation reproducible

for(i in 1:10000){                     # we'll do this 10k times
x1 = rnorm(N)
x2 = rnorm(N)                        # these variables are uncorrelated
x3 = rnorm(N)
y  = beta0 + beta1*x1 + beta2*x2 + beta3*x3 + rnorm(100)
mod = lm(y~x1+x2)                    # note all 3 variables are relevant
# but the model omits x3
b0VectU[i] = coef(mod)[1]            # here I'm storing the estimates
b1VectU[i] = coef(mod)[2]
b2VectU[i] = coef(mod)[3]
}
mean(b0VectU)  # [1] -71.00005         # all 3 of these are centered on the
mean(b1VectU)  # [1] 0.8399306         # the true values / are unbiased
mean(b2VectU)  # [1] 0.6398391         # e.g., .64 = .64

############## correlated version

r23 = .7                               # this will be the correlation in the
b0VectC = vector(length=10000)         # population between x2 & x3
b1VectC = vector(length=10000)
b2VectC = vector(length=10000)
set.seed(2734)

for(i in 1:10000){
x1 = rnorm(N)
X  = mvrnorm(N, mu=c(0,0), Sigma=rbind(c(  1, r23),
c(r23,   1)))
x2 = X[,1]
x3 = X[,2]                           # x3 is correated w/ x2, but not x1
y  = beta0 + beta1*x1 + beta2*x2 + beta3*x3 + rnorm(100)
# once again, all 3 variables are relevant
mod = lm(y~x1+x2)                    # but the model omits x3
b0VectC[i] = coef(mod)[1]
b1VectC[i] = coef(mod)[2]            # we store the estimates again
b2VectC[i] = coef(mod)[3]
}
mean(b0VectC)  # [1] -70.99916         # the 1st 2 are unbiased
mean(b1VectC)  # [1] 0.8409656         # but the sampling dist of x2 is biased
mean(b2VectC)  # [1] 0.8784184         # .88 not equal to .64

• So, can you explain a bit more - what happens if we assume that $x_3$ is not correlated with $x_1$ and $x_2$? Then what happens if I estimate $y=a+b_1x_1+b_2x_2+u$? – renathy May 11 '13 at 15:04
• $b_3x_3$ will be incorporated into the residuals either way, but if it is uncorrelated in the population, then your other $b$s won't be biased by the absence of $x_3$, but if it's not uncorrelated, then they will be. – gung - Reinstate Monica May 11 '13 at 15:08
• To state this more clearly: If $x_3$ is not correlated with either $x_1$ or $x_2$, you are OK. – gung - Reinstate Monica May 11 '13 at 15:10
• I discuss the flip side of this issue in my answer here: Does adding more variables into a multivariable regression change coefficients of existing variables? – gung - Reinstate Monica Sep 18 '14 at 0:24

Let's think of this in geometric terms. Think of a "ball", the surface of a ball. It is described as $r^2 = ax^2+by^2+cz^2 + \epsilon$. Now if you have the values for $x^2$, $y^2$, $z^2$, and you have measurements of $r^2$ then you can determine your coefficients "a", "b", and "c". (You could call it ellipsoid, but to call it a ball is simpler.)

If you have only the $x^2$, and $y^2$ terms then you can make a circle. Instead of defining the surface of a ball, you will describe a filled in circle. The equation you instead fit is $r^2 \le ax^2 + by^2 + \epsilon$.

You are projecting the "ball", whatever shape it is, into the expression for the circle. It could be a diagonally oriented "ball" that is shaped more like a sewing needle, and so the $z$ components utterly wreck the estimates of the two axes. It could be a ball that looks like a nearly crushed m&m where the coin-axes are "x" and "y", and there is zero projection. You can't know which it is without the "$z$" information.

That last paragraph was talking about a "pure information" case and didn't account for the noise. Real world measurements have the signal with noise. The noise along the perimeter that is aligned to the axes is going to have a much stronger impact on your fit. Even though you have the same number of samples, you are going to have more uncertainty in your parameter estimates. If it is a different equation than this simple linear axis-oriented case, then things can go "pear shaped". Your current equations are plane-shaped, so instead of having a bound (the surface of the ball), the z-data might just go all over the map - projection could be a serious problem.

Is it okay to model? That is a judgment call. An expert who understands the particulars of the problem might answer that. I don't know if someone can give a good answer if they are far from the problem.

You do lose several good things, including certainty in parameter estimates, and the nature of the model being transformed.

The estimate for $b_3$ disappears into epsilon and into the other parameter estimates. It is subsumed by the whole equation, depending on the underlying system.

• I can't really follow your argument here, & I'm not sure if it's correct. E.g., the surface area of a sphere is $4\pi r^2$. Beyond that, I'm not sure how this relates to the question. The key issue is whether or not the omitted variable is correlated w/ variables that are in the model. I'm not sure how what you are saying addresses that issue. (For clarity, I demonstrate this with a simple R simulation.) – gung - Reinstate Monica May 11 '13 at 23:08
• Gung. I gave a best-case answer sphere -> circle and showed that it changed the model in unexpected ways. I liked the technical sophistication of your answer, but am not convinced that the asker is able to use either of our answers. the $f(x,y,z)$ is the equation for the surface of an ellipsoid in 3 dimensions, a sphere is one case of it. I am assuming that the "true model" is the surface of the sphere, but noise corrupted measurements are on the surface. Throwing out one dimension gives data that, at best, makes a filled circle instead of the surface of a sphere. – EngrStudent May 12 '13 at 4:55
• I am unable to follow your argument because I don't see anything that corresponds to a "filled in square." – whuber Sep 3 '18 at 20:19

The other answers, while not wrong, over complicate the issue a bit.

If $x_3$ is truly uncorrelated with $x_1$ and $x_2$ (and the true relationship is as specified) then you can estimate your second equation without an issue. As you suggest, $\beta_3 x_3$ will be absorbed by the (new) error term. The OLS estimates will be unbiased, as long as all the other OLS assumptions hold.