7
$\begingroup$

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, 7th ed., p. 58).

The other day I stumbled upon Judea Pearl's treatment of Simpson's Paradox and a nice webpage that simulates how "stepwise inclusion of control variables into a regression model switches the sign of an estimated causal association in every step". To me, this somehow contradicts the above statement. I feel this could be a very subtle (though incredibly important) problem, so any pointer to further literature would be very helpful. What especially strikes me is that Greene claims he has a proof for his assessment.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
10
$\begingroup$

There's no contradiction.

The first paragraph there talks about superfluous variables.

If Simpson's paradox applies, the variables are not superfluous.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ In the problem as posed on the website, if one adjusts for for Z1 and Z2, the estimate is biased. Z1 seems indeed not to be superfluous, but what about Z2? By construction, it does not affect either X or Y, yet its inclusion biases the estimate. $\endgroup$
    – Julian Schuessler
    Oct 20, 2014 at 9:31
  • 2
    $\begingroup$ Depending on the exact relations between these variables, a superfluous variable with extremely high correlation with one of the other independent variables can lead to sign reversals. This is also covered in the Greene book in the part about multicollinearity. He states that high levels of multicollinearity can lead to unstable and unreliable coefficients because of the near singularity. $\endgroup$
    – Andy
    Oct 20, 2014 at 9:59
  • $\begingroup$ I should have mentioned that the previous comment was more for @JulianSchuessler. For Glen_b's answer +1 $\endgroup$
    – Andy
    Oct 20, 2014 at 11:06
  • 3
    $\begingroup$ Z2 does not cause X or Y, but it is $d$-connected to X via the unobserved variable U, and to Y via Z3. So it is correlated with both X and Y. If you define "superfluous" as "independent" then Greene is correct - conditioning on a variable Z independent of X and Y will not bias your estimate (excluding cases where the independence is "unfaithful" to the causal relationships). I think multicollinearity is a separate issue. Bias from conditioning on "collider" variables does not require very high dependence between the variables, and doesn't blow up the variance of your estimate. $\endgroup$
    – Lizzie Silver
    Oct 20, 2014 at 22:30
  • $\begingroup$ @LizzieSilver: Thanks, this is also my current understanding, having looked deeper into Pearl's work: If one blocks all backdoor-paths by including the appropriate regressors, one gets an unbiased estimates. However, it is also absolutely clear from Pearl's work that adjusting for the wrong variables, which might be correlated with both X and Y, biases the causal effect estimate of the variable of interest. So I wonder what to make of the usual proof of unbiasedness. Maybe the wrong regression is unbiased, but the coefficient in it does not equal the causal effects but something else? $\endgroup$
    – Julian Schuessler
    Jan 31, 2015 at 14:13
5
$\begingroup$

Consider a postulated linear regression model

$$y_i = b_0 + b_1X_{1i} + b_2X_{2i} + u_i,\;\; i=1,...,n$$

As a matter of algebra (and not any stochastic assumptions), the OLS estimator in matrix notation is

$$\hat b = b + \left(\mathbf X'\mathbf X\right)^{-1}\mathbf X'\mathbf u$$

Its expected value conditional on the regressor matrix is therefore

$$E\left(\hat b\mid \mathbf X\right) = b + \left(\mathbf X'\mathbf X\right)^{-1}\mathbf X'E\left(\mathbf u\mid\mathbf X \right)$$

So: If "strict exogeneity" of the regressors with respect to the error term holds, or, in other words, if all error terms are mean-independent from all regressors,past present and future, (which is the benchmark assumption in the Classical Linear Regression model), i.e. if $E\left(\mathbf u\mid\mathbf X \right)=\mathbf 0$, we will have

$$E\left(\hat b\mid \mathbf X\right) = b + \mathbf 0 \Rightarrow E(\hat b) = b$$

using also the law of iterated expectations.

Given all the above, what "superfluous variable" means? I take it, it means "unrelated" to the dependent variable. But "unrelated" should be translated as "stochastically independent". But if it is independent from the dependent variable, it is necessarily independent from the error term (and so also strictly exogenous with respect to it), so all the above hold for any superfluous variable also, and the OLS estimator is unbiased even if, say, the variable $X_2$ is "superfluous" and the true model does not contain it.

This is how econometricians understand the issue. Now, in a more general setting, "superfluous" could mean that say, $X_2$ is independent of $y$ conditional on the presence of $X_1$ (which I suspect is more close to what Pearl has in mind). Still, as long as $X_2$ is strictly exogenous to the error term, the unbiasedness result holds.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.