1
$\begingroup$

As far as scaling, Wikipedia says:

Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF).

But that means that all the variables would need to be scaled by the same constant, which is not how feature scaling is usually done.

$\endgroup$
0
$\begingroup$

Answer is a combination of my knowledge and recommendations from RekhaMolala on Medium:

To reiterate, VIF is calculated as 1/(1-R^2); R^2 (R-square) is obtained from the OLS model fitted for each numeric feature as a dependent variable and rest all features (leaving the actual target variable) as predictors. R^2 of say 0.65 implies that the predictors used in the OLS model can explain the linear behavior of the dependent variable up to 65%.

So, think of all the data processing that you do before calculating VIF as data processing that you would do before fitting a non-regularized linear regression model.

Scaling

There is some debate about when scaling is needed, but in the case where you create interaction features (multiplying 2+ features together), scaling is recommended. Scaling can help reveal reliable p-values of the features in presence of multi-collinear features (especially in the presence of interaction features).

Missing values

Linear regression packages will either drop rows containing missing values, or return an error. Therefore, missing values need to be dealt with in some way. That's a whole other topic. Dropping too many rows will result in too small of a sample size to be representative of the whole sample, and could introduce bias into the distribution of the sample if the data is MAR or MNAR. Current recommended best practice is to use MICE (multiple imputation), or KNN imputation if MICE will take too long. Mean imputation has the poorest performance of imputation methods, but is the simplest and fastest.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.