2
$\begingroup$

I have a dataset that contains 292 attributes. I want to calculate VIF to address multicollinearity in dataset. Here is a snippet which I have used to calculate VIF

from statsmodels.stats.outliers_influence import variance_inflation_factor  
def calculate_vif_(X):
    variables = list(X.columns)
    vif = {variable:variance_inflation_factor(exog=X.values, exog_idx=ix) for ix,variable in enumerate(list(X.columns))}
    return vif

calculate_vif_(train)

Here is the example of output

    {'0_13_all': inf,
     '0_13_female': inf,
     '0_13_male': inf,
     '0_17_all': inf,
     '0_17_female': inf,
     '0_17_male': inf,
     '0_6_all': inf,
     '0_6_female': inf,
     '0_6_male': inf,
     'ID_big_road1': 4.3585189697040807,
 'ID_big_road2': 3.3490088398280924,
 'ID_bus_terminal': 8.4094163765196654,
 'ID_metro': 4.3989179012404227,
 'ID_railroad_station_avto': 6.1953273896236496,
 'ID_railroad_station_walk': 6.765644767428685,

}

The dataset contains only numeric variables and I have removed all the NA's. I couldnt figure out why some of the variables have inf as output. Is there any other cleaner way to address multicollinearity?

$\endgroup$
5
  • $\begingroup$ De-bugging your code is off topic here but there does seem to be a possible statistics question here. Is all=female+male? $\endgroup$ – mdewey May 6 '17 at 14:38
  • $\begingroup$ You can't include both female and male. And, of course, those are not continuous variables. $\endgroup$ – Peter Flom May 6 '17 at 15:17
  • 1
    $\begingroup$ These VIFs tell you there is perfect collinearity: you have completely redundant variables. The very first thing you should do to address collinearity is to think about what the variables mean. Unless you are coding for more than two genders, for instance, including an indicator of maleness is completely redundant with an indicator of femaleness. $\endgroup$ – whuber May 6 '17 at 16:21
  • $\begingroup$ None of the variables are cateogorical in nature. they are of numerical type...Does that mean I have high collinearity between variable? $\endgroup$ – Vivek Srinivasan May 6 '17 at 17:18
  • $\begingroup$ Even I'm addressing a similar issue with the classic Melbourne house prediction problem, I have one hot encoded few nominal features and their VIF is inf. My question is since there will be a good correlation between one-hot-encoded columns of the same categorical feature (for instance if Gender is one hot encoded as Gen_Male and Gen_Female columns, they will have a very high correlation). Should I let them stay in the regression model or remove them because their vif = inf?? Thanks in Advance :D 0 9.941227 Rooms 1 1.124820 Date 2 2.128941 Distance 3 9.451651 Bedroom2 . . . . 18 inf R_name_Eas $\endgroup$ – Shhiv Jan 14 '19 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.