0
$\begingroup$

I have a binary response variable (presence/absence) and four independent variables (min.temp, max.temp, precipitation and elevation. My scatter matrix is showing collinearity between 3 of the variables with cor.efficients of 0.73, 0.74, 0.84. ELevation however doesn't seem correlated with the rest. I therefore thought it was best to run binary logistic regression on each variable seperately with elevation? However, when I checked the vif Score of the model when all were included it was only showing min.temp as high (9.13) and max temp (4.7) so it appeared I could include all but min.temp....I'm confused.

$\endgroup$
8
  • 1
    $\begingroup$ Welcome to Cross Validated! Why not include all of the variables? $\endgroup$
    – Dave
    Feb 7, 2023 at 15:52
  • 5
    $\begingroup$ Better title could be framed, I guess, to articulate your query. $\endgroup$ Feb 7, 2023 at 15:52
  • 2
    $\begingroup$ I'm not convinced that you need to do anything at all to address multicollinearity, and I'm definitely not convinced that the potential bias introduced by dropping a variable is worth the reduced variance in estimating what remains. $\endgroup$
    – Dave
    Feb 7, 2023 at 17:44
  • 1
    $\begingroup$ It sure seems like you have collinearity. Why do you see that to be a problem? I discuss here some of the misunderstandings surrounding this topic. $\endgroup$
    – Dave
    Feb 7, 2023 at 18:35
  • 1
    $\begingroup$ 93 sample points, 78 present, 15 absent $\endgroup$ Feb 7, 2023 at 20:07

1 Answer 1

2
$\begingroup$

There's a lot of information on the site about multicollinearity, it may benefit you to click on the tag, sort by votes or frequently cited, and start reading to learn about the topic.

In general, multicollinearity means that some of your variables 'point' in the same direction / aren't distinct from each other. As a result, it's hard for the model to know which is associated with the response. That doesn't necessarily impair the estimates of the coefficients, but it makes the estimates much less certain. That is, the standard errors, CIs, etc., will be much wider. It means you'll have less power than you otherwise would have, although you still might get significant results (see my answer to: What is the effect of having correlated predictors in a multiple regression model?).

Regarding how to detect collinearity, the VIF is superior to looking at marginal correlations. By convention, collinearity is problematic if the VIF is greater than 10. Your largest VIF is lower than that, so by this convention, it isn't a problem. If there were a problem, it would only mean that you might get a non-significant result even though there was a real association. A way to deal with that would be to drop all three strongly related variables and perform a nested model test. The multicollinearity wouldn't affect that test. If it were significant, then you might be confident that at least one of those variables was related, but without more information, it would be difficult to know which.

None of that is really your problem, though. The biggest issue here is that there is very little information in binary data (i.e., presence/absence). A basic rule of thumb is that you need at least 15 of the less commonly occurring category (here, absent) for each variable you want to include. You might have enough information for a single variable, but not four. As an exploratory analysis, you could run four different univariate logistic regressions, but it's sort of cheating. It might provide some information about where to look further, but you wouldn't want to draw any firm conclusions.

$\endgroup$
3
  • $\begingroup$ Thankyou, that's all very helpful. I didn't choose the dataset unfortunately. When I have run the four univariate regressions elevation has come up as the most statistically significant. Sorry for being dumb, so you are basically saying I should be ok to run it as a whole? All four variables included also shows as statistically significant .. $\endgroup$ Feb 7, 2023 at 21:28
  • $\begingroup$ It's a rule of thumb. I'd be quite wary of running a model with 4 variables where you only have 15 absents, but there's no law against it. The police aren't going to show up at your door. Good luck. $\endgroup$ Feb 7, 2023 at 21:44
  • $\begingroup$ Thankyou so much. $\endgroup$ Feb 7, 2023 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.