1
$\begingroup$

Consider the output from two different r packages, rms::vif and car::vif:

model <- glm(Survived ~ ., family = "binomial", data = Titanic)

rms::vif(model)
 Class2nd  Class3rd ClassCrew SexFemale  AgeAdult      Freq 
 1.482212  1.533078  1.474158  1.118509  1.426962  1.571872 

car::vif(model)
          GVIF Df GVIF^(1/(2*Df))
Class 1.063587  3        1.010327
Sex   1.118509  1        1.057596
Age   1.426962  1        1.194555
Freq  1.571872  1        1.253743

I believe the typical multicollinearity procedure is as follows:

  1. Calculate VIF for the model.
  2. Identify vars with VIF > 5; remove one at a time (highest first); re-check VIF and repeat procedure until VIF(all_vars) < 5.

However, does this procedure differ in the case of calculating VIF for each level of a factor variable (e.g. Crew here has four levels: 1st, 2nd, 3rd, and Crew)?

Let's pretend the output of rms::vif was

rms::vif(model)
 Class2nd  Class3rd ClassCrew SexFemale  AgeAdult      Freq 
 5.482212  1.533078  1.474158  1.118509  1.426962  1.571872

Do we still remove Class entirely or is better to just remove the level Class_2nd and then re-check VIF?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Item 3. in this post will come in handy: statisticalhorizons.com/multicollinearity. $\endgroup$
    – Isabella Ghement
    Jan 24, 2019 at 2:27
  • 1
    $\begingroup$ @IsabellaGhement thank you for the reference. If I am interpreting that correctly, this means the output of rms::vif() will always be dependent on what the reference level is for the factor variable, but the output of car::vif() will be unchanged. However, it is still unclear (at least to me) whether or not we should remove a level or if looking at by-level VIFs has any merit in the decision to remove or include the factor as a whole, re-group the factor, or remove one or more levels. In addition, it begs the question of what's the point of looking at by-level VIF such as rms::vif()? $\endgroup$
    – JasonAizkalns
    Jan 24, 2019 at 16:06
  • $\begingroup$ You're welcome, Jason! That's exactly what I understand, but you can double check what rms::vif() does by releveling the Class factor with the relevel(Titanic$Class, ref = "Crew") and re-fitting the model. $\endgroup$
    – Isabella Ghement
    Jan 24, 2019 at 17:32
  • $\begingroup$ You should not remove a level - all levels are used as a set to encode the effect of the categorical variable Class on whether or not a passenger survived, given the other predictors in the model. If the GVIF is too large, you can exclude Class altogether from your model. $\endgroup$
    – Isabella Ghement
    Jan 24, 2019 at 17:35
  • $\begingroup$ The impact of collinearity in your model depends on the purpose of the model. If the model is used for explanation rather than prediction, then collinearity is more of an issue. $\endgroup$
    – Isabella Ghement
    Jan 24, 2019 at 17:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.