# Avoiding multicollinearity in logistic regression

I have a dataset with variables for both the country (say, US & Canada) and region/state (say, CA, TX, ON, & BC). I want to estimate the effect of location on a binary response variable (like an admission to college).

Assuming that the values for CA & TX are similar to one another, and those for ON & BC the same, multicollinearity becomes an issue. An easy solution is to drop the country predictor variable from the model.

However, what if the effect of the country is substantively important and I want the reader to know both the effect of the country and that of the region/state? Should I use two separate logit models?

• That is a good question. I wonder how you use the variables in your model? Most models need numeric variables as input. Your variables are categorical, so the best way to deal with them would be to transform them to binary variables for each country/region. Maybe you can expand on that a little bit. Are you using an encoding for your regions e.g. CA=1, BC=2 or are you generating binary variables? – Sören Jun 14 '18 at 16:05
• The variables are categorical/discrete, which is not a problem for logistic regression per se (certainly not in R). Recoding is easy to do if necessary, but I'm not sure how it would help with multicollinearity. – KaC Jun 14 '18 at 16:07
• Are state/province and country your only variables in this regression? – Noah Jun 14 '18 at 16:26
• In my actual dataset, no. I also have variables like gender (also categorical) and age (continuous). But I'd like to know how to deal with situations like this for different kinds of models for future reference. – KaC Jun 14 '18 at 16:30

I think the main multicollinearity problem is that country is completely predictable from region/state. Say your dummy variables are I(US), I(CA), I(TX) and I(ON) (where I(US) is the indicator variable: I(US) = 1 if that data point comes from the US and I(CA) = 1if it comes from California, etc.) Then, in this case I(US) = I(CA) + I(TX), which is perfect collinearity.

I think you can drop country, then run a post-hoc test on the contrast between the average Canadian and average American state. Here's an example with the multcomp package:

library(multcomp)
samples = list()
samples[["ca"]] = cbind("ca",rbinom(100,1,.6))
samples[["tx"]] = cbind("tx",rbinom(100,1,.5))
samples[["on"]] = cbind("on",rbinom(100,1,.85))
samples[["bc"]] = cbind("bc",rbinom(100,1,.75))
data = rbind.data.frame(samples[["ca"]],samples[["tx"]],samples[["on"]],samples[["bc"]])
colnames(data) = c("region","response")

model = glm(response ~ region, family = binomial, data)
contrast = glht(model, linfct=t(c(0,.5,-.5,-.5)), alternative = "less", rhs=0)
summary(contrast)

• This is exactly what I would have suggested. One additional thing to note is that averaging the countries with equal weight may not be the most meaningful comparison if certain states are more relevant to the average than others (e.g., down-weighting RI and up-weighting TX, perhaps based on population, size, or GSP). Also, I would expect that using the equal weights method that you would arrive at the same results as doing a regression on just the countries. – Noah Jun 14 '18 at 17:25
• This makes sense for a model with fixed effects. In some contexts, however, it might make sense (or provide more power) to treat the provinces/states as random effects instead. In that case you could devise a nested model, with state/province nested within country. Then you could include a fixed predictor term for country without running into such a problem. – EdM Jun 14 '18 at 17:42
• @EdM I actually thought about that, but I figured that in the OP's data all the states/provinces are likely present, in which case random effects seem not to be needed. – WavesWashSands Jun 14 '18 at 17:45
• I was thinking about situations where you model interaction terms and are more interested in between-country differences than in predictions for individual states. That might be more efficiently modeled with random effects, even if all of the states are included. – EdM Jun 14 '18 at 18:01