3
$\begingroup$

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to statistics for some of my modeling routines.

I’m attempting to predict habitat suitability for a given species with a resource selection function using a binary logistic regression model. My model employs multi-scale explanatory variables that quantify various resources in the surrounding environment. For example, when considering forested cover, I’ve created three potential explanatory variables; 1) % cover within 100-meters (local scale), 2) % cover within 350-meters (intermediate scale) and 3) % cover within 1-km (landscape scale). In total, I have 31 possible predictors.

In similar past models, I’ve dealt with the issue of correlation by examining Pearson correlation coef’s; in cases where two parameters were correlated, I’d perform univariate analysis on each variable in question to see which had the greatest explanatory value by looking at the resulting model AIC scores.

In this recent model, I followed the same approach and produced a model that seemed to have good predictive ability. However, a colleague questioned the method I used for dealing with correlated variables and indicated I should be more concerned with collinearity; I had thought that correlation equates to collinearity, but my recent research indicates that that correlation is necessary for collinearity, but does not automatically equate to collinearity. To that end, he suggest examining the VIF’s of the explanatory variables I had used in the final model and, sure enough, two of the variables that were hypothesized to be important predictors had high VIF’s, around 14+ for one and 18+ for the other.

So, my questions concern correlation/collinearity among the possible predictors. First, when examingin the variables, should I be concerned with correlation or simply with collinearity, or possibly both? Secondly, what are the best methods for dealing correlated/collinear predictors? I’ve seen methods similar to my approach above, and also another similar approach whereby VIF’s are examined and dropped iteratively until all predictors have satisfactory VIF’s. I’ve also seen variable clustering/reductions methods (ie. PCA), shrinkage methods (i.e. LASSO), etc., but I have little experience with those methods and not sure what the pros/cons are of employing those methods.

My main objective right now is to select the best set of predictors while minimizing loss of information and maintaining variable explanation, hopefully resulting in a logistic model that is not over-fit with good predictive ability. I apologize for the open-ended nature of this and realize there are likely numerous ways to deal with this, but hoping someone with solid knowledge in this arena can enlighten myself and start to steer me in the right direction. Thanks to all in advance.

$\endgroup$

1 Answer 1

1
$\begingroup$

You wrote:

My recent research indicates that that correlation is necessary for collinearity, but does not automatically equate to collinearity. To that end, he suggest examining the VIF’s of the explanatory variables I had used in the final model

Actually, you can have low correlation and still have collinearity. A made up example is where you have (say) 10 independent variables, none of them correlated, and one more IV that is the sum of the other 10. There will be perfect collinearity but all the correlations will be modest.

VIFs are OK as a tool, but I prefer condition indexes, I wrote my dissertation abut this, or see books by David Belsley.

First, when examingin the variables, should I be concerned with correlation or simply with collinearity, or possibly both?

Collinearity is the problem for regression methods.

Secondly, what are the best methods for dealing correlated/collinear predictors?

It depends on your goals. Methods include:

  • Dropping variables
  • Doing a principal component analysis or factor analysis and using the resulting component or factor as an IV
  • Ridge regression or elastic net

and maybe other things as well

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.