0
$\begingroup$

I came across an article with the following question:

Consider a linear regression with one single covariate, y=β0+β1x1+ε and the least-square estimates. The variance of the slope is Var[β1]. Do we decrease this variance if we add one variable, and consider y=β0+β1x1+β2x2+ε ?

They showed, using simulation, that the answer is no as it depends on how correlated x1 and x2 are. Is there a way to prove this without simulation?

$\endgroup$
  • $\begingroup$ Take a look at the VIF ( variance inflation factor) it may be useful to understand the idea behind this result. $\endgroup$ – RScrlli Mar 27 '19 at 21:53
  • $\begingroup$ What happens to the variance when $x_2$ is a constant nonzero multiple of $x_1$? $\endgroup$ – whuber Mar 27 '19 at 22:27
0
$\begingroup$

Consider the text book definition for the sampling variances of the OLS slope estimators for covariate j:

$$ Var(\hat{\beta_j}) = \frac{\sigma^2}{SST_j(1-R_j^2)} $$

where $\sigma^2$ is the variance of the error term, SST is the total sample variation $\sum_{i=1}^n(X_{ij}-\bar{X}_j)$ in $X_j$ and $R_j^2$ is the R-squared from regressing $X_j$ on all other independent variables.

Coming back to your example it's easy to see that the variance of $\hat{\beta_1}$ increases when the correlation between $X_{1}$ and $X_{2}$ increases because a higher correlation implies a higher R-squared from regressing $X_{1}$ on $X_{2}$ (the denominator decreases).

Importantly, a larger variance of your estimator translates into larger confidence intervals and more uncertainty.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.