A regression specification problem: what if one control variable is a function of another—does this cause any issues?

Suppose you run a regression:

$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \epsilon_i$

but you believe that:

$x_{i1} = f(x_{i2})$

will this cause any issues for your estimation and interpretation of the regression coefficients?

The following source (http://www.swlearning.com/pdfs/chapter/0324289782_3.PDF) on p. 84 states that in a simple regression setting your estimated beta coefficient “measures the sample relationship between $y$ and $x_1$ after $x_2$ has been partialled out” (removed). My concern, more generally, is that if you have two controls that pick up much of the same variation of $y$—and you choose to include both—then the general effect that they both try to identify upon $y$ may show up as insignificant (since it is—crudely put—split across the two), this while only including one of the two controls could have resulted in a significant identification of the general effect.

Am I wrong in thinking along these lines? I recognise that omitting a variable that is correlated with both $y$ and a control $x$ results in an omitted variable problem/bias. I am, however, not sure if the omitted variable problem is still a “bias inducing problem” when we believe that e.g. $x_{i1} = f(x_{i2})$ holds.

Any thoughts/comments would be much appreciated.

PS: I note that there are several questions on regression specification and general insignificance, but I have not found one that directly addressed my concerns.

• I think you are talking about polynomial regression, look at the wikipedia page and many many posts here about polynomial and quadratic for a special common case. – Steve Sep 30 '14 at 18:35
• Reading some more it appears that this is a classic problem of multicollinearity (see: en.wikipedia.org/wiki/Multicollinearity). – Seb Sep 30 '14 at 19:21
• Sounds more like a two-stages least squares or structural equation model problem. – Corey Sparks Oct 2 '14 at 2:35

If $X_1$ is an exact affine function of $X_2$, $X_1 = \alpha_0+\alpha_1X_2$, then the "problem" is that your regressor matrix will be singular and non-invertible, and so OLS estimation will break down. But there is a way out, since in reality in such a case you have

$$y= \beta_0 + \beta_1(\alpha_0+\alpha_1x_2) + \beta_2x_2 + u$$

$$\Rightarrow y = \gamma_0 + \gamma_1x_2 + u,\;\; \gamma_0 = \beta_0+\beta_1\alpha_0,\;\; \gamma_1=\beta_1\alpha_1+\beta_2$$

In other words, in such a case you can run a simple regression with only $X_2$ present as regressor, and you will obtain estimates for the $\gamma$ coefficients.

If $f(X_2)$ is not a linear function, in general, no problem arises. In fact very often one sees $f(X_2) = X_2^2$, i.e a regression of the form

$$y= \beta_0 + \beta_1x_2^2 + \beta_2x_2 + u$$

Remember, whether the regression is "linear or not" has to do with whether it is linear in the unknown coefficients, not in the regressors.

• Thanks Alecos. That is a nice intuitive explanation. As I mentioned in a comment above this seems to be the standard issue of multicollinearity (when f(.) is taken to be linear). However, programs like Stata (to my knowledge) would often give a warning (and drop the variable causing issues) when this happens--I assume due to the invertability issue you mentioned. However, this is not the case in my application. With that said, what is the practical solution here? Do I drop one of the $x's$ from the regression in your example? – Seb Sep 30 '14 at 19:31
• When you say "this is not the case in my application", do you mean "in my case, the $f()$ function is not linear" or "$f()$ is linear, but the software does not give any warning"? And if you mean the second, then how exactly does your software react? – Alecos Papadopoulos Sep 30 '14 at 20:02
• Sorry for not being clear. I mean that f() is not linear since that would have been picked up by the software. I read some more about it and it seems like the problem is a bit like double edged sword in that there is a tradeoff--if I remove a variable then I increase the risk of biased coefficient estimates, while if I keep it in my coefficient estimates will be precise but the standard errors will be large (possibly yielding statistical insignificance where it would otherwise yield significance).I guess my question is: what do people do in practice to resolve this issue? – Seb Oct 1 '14 at 13:40