5
$\begingroup$

What is the difference between the variance inflation factor (VIF) and stepwise regression as both help in detecting multicollinearity? What variables are different while running both techniques?

$\endgroup$

2 Answers 2

3
$\begingroup$

VIF and stepwise regression are two different beasts. Stepwise regression is an exercise in model-building, whereas computing VIF is a diagnostic tool done post-estimation to check for multicollinearity. Therefore, there is no answer to the second part of your question ("What variables are different while running both techniques?"), because VIF is not a model-building technique.

With stepwise regression, you are either adding (forward) or deleting (backward) variables from the model and seeing how estimates change. Typically, variables are "kicked out" of the model if the p-values do not cross a certain threshold pre-set by the researcher (e.g. if $p>0.10$).

VIF is done when you already have a model to work with. Calculation of VIF is fairly straightforward. Given the model:

$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 +\beta_3X_3 +\beta_4X_4 +\epsilon$

You can calculate the VIF of each parameter estimate $i$ (e.g. $\hat\beta_1$,$\hat\beta_2$, $...$ ,$\hat\beta_i$) using the formula $VIF_i = 1/(1-R_i^2)$ where $R_i^2$ is the $R^2$ from a model predicting $X_i$ using all other covariates as predictors, e.g.,

$X_1 = \delta_0 + \delta_2X_2 +\delta_3X_3 +\delta_4X_4 +\nu$

$\endgroup$
1
  • 3
    $\begingroup$ I don't think VIF if necessarily done post-estimation. Do you have a source for this? I see a lot of papers and blog posts where VIF is used to screen highly correlated variables. $\endgroup$
    – skeller88
    Apr 17, 2020 at 20:00
1
$\begingroup$

They aren't really that similar. Stepwise regression is a technique for finding a subset of variables that are useful in predicting a response. It is a very old and intuitive strategy. Unfortunately, it is not as good as it seems (see here). In addition, it is not for detecting multicollinearity.

The variance inflation factor can be used to assess multicollinearity. Multicollinearity refers to the fact that your predictor variables are correlated. When just two variables are collinear, it is easy to see. But collinearity can exist amongst several variables (hence 'multi-'), which is harder to detect. If you were to run a multiple regression in which one of your X variables were the response and your other X variables were taken as predictors, you would hope to find a multiple $R^2=0$. That would mean that they are perfectly uncorrelated. With observational data that is very uncommon, though. As the $R^2_j$ for each of your variables goes up, the degree of multicollinearity increases. This will cause your standard errors to be larger than they would otherwise have been (if your X variables had been perfectly uncorrelated). To find out how much larger the variance of the sampling distribution for the $j^{\rm th}$ variable is, you can check the VIF. You calculate the VIF as so:
$$ {\rm VIF} = \frac{1}{1-R^2_j} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.