2
$\begingroup$

I was wondering if, in the case where I use the same set of predictors for different kind of models (e.g. ANOVA, Poisson GLM, logistic regression...), VIF values for each predictor would be similar across models or if they would vary depending on response variables and model types (e.g. logistic vs. Poisson GLM)?
EDIT: the helpful comments below confirmed my idea that, since correlations among predictors are independent of response variables, VIF values should be similar across model types. However, if I understand explanations from EdM correctly (in the comments as well as here and here), the actual effects of predictors multicollinearity on coefficients will vary between models fitted with ordinary least squares and those using Maximum Likelihood. Consequently, the use of generalized VIF (GVIF; see also here) or Condition Indexes for each model is recommended.


Still, I am unsure about the two following things:

  • Does that mean that VIF/GVIF values would be indifferent to model misspecification/overfitting? I'd figure that it is perhaps not a good idea to compute correlations with an inappropriate sample size. To illustrate my questioning with a ridiculous example, would it be okay to compute GVIF for 30 predictors in a model with only 50 observations?

  • Does that make any sense to preventively drop redundant predictors before actually assessing the effects of this multicollinearity on coefficient estimates?

$\endgroup$
5
  • 2
    $\begingroup$ Multicollinearity is unrelated to the response variable, so whatever method you use for one will be valid for the others. $\endgroup$
    – Dave
    Commented Jan 19, 2021 at 15:18
  • 4
    $\begingroup$ Does this answer your question? In plain language, why is there no VIF for binary outcome regression models? You need to distinguish between multicollinearity among the predictors and the way that the multicollinearity plays out in terms of the covariances among the coefficient estimates. With ordinary least squares those are the same; with models fit by ML, you might need to examine the model-based covariance matrix more directly, which will differ among model types. $\endgroup$
    – EdM
    Commented Jan 19, 2021 at 16:03
  • $\begingroup$ @EdM, your helpful link and comment greatly answers my first question, but I do not see how it answers my second question (but there may be something I'm too bad at statistics to see). I will thus edit my question to account for your comment and improve the clarity of my questioning. $\endgroup$
    – Fanfoué
    Commented Jan 20, 2021 at 18:40
  • $\begingroup$ Thanks for editing your question to be more specific. What do you wish to accomplish by "assessing the effects of multicollinearity"? See, for example, the discussion on Is multicollinearity really a problem? In particular, are you primarily interested in a predictive model or are you trying to do inference or make causal interpretations? $\endgroup$
    – EdM
    Commented Jan 20, 2021 at 21:19
  • $\begingroup$ My interest is to do inference, I would like to "explain" the variation of my response variables by selecting (on the basis of AICc) the "best" models that I will build a priori based on hypotheses relevant in my field (ecology). According to Zuur et al. (2009), dropping collinear variables is important to ensure good inference. Additionally, as I have too many explanatory variables (and hypotheses) for my limited sample size, multicollinearity is a criteria for me to simplify my models. $\endgroup$
    – Fanfoué
    Commented Jan 21, 2021 at 15:33

0