# Logistic regression assumptions for a model with many binary independent variables

I am working on developing a logistic regression model that uses qualitative variables only ($n=990$). My remit is to define the equation that can identify the most relevant characteristics of a survey respondent that is favorable towards Company X. The proposed equation is similar to the following: \begin{align} {\rm Fav} = &a*{\rm age} + b*{\rm CoAware} + c*{\rm IssueAware} + d*{\rm readnewspaper} + e*{\rm region} + \\ &f*{\rm income}\ldots \end{align} The dependent variable is "Company Favorability" (0 = Unfavorable/Neither | 1 = Favorable). There are currently 25 independent variables, 20 of which are binary IVs that range from highly correlated to the DV (awareness of Company) to not significant (gender). I also have 5 categorical variables that indicate region of the country, age (in categories), party affiliation, income level, and education.

I am almost certain that I need to use a logistic regression model for this approach. However, when I test my assumptions, I have having a very difficult time proving a linear relationship between the dichotomous independent variables and the logit transformation of the DV.

My other problem is that, I am somewhat overwhelmed by possible interaction effects. There are 34 possible options using 25 variables - leading me to over 50 million possible combinations.

I have three questions:

1. Is there a better method to model with a binary dependent variable?
2. Am I missing something in the assumptions? (ie: Do I need to indeed prove the linear relationship if all of my variables are dichotomous)
3. Would it be better to approach this by looking at multicollinearity first, to reduce the number of variables overall, and then look at linear relationships with the logit of the DV?
• I admire the work and the hard thinking that this question reflects (+1). You're concerned with many important issues with which many of us struggle. Unfortunately I think the answers will come not from a few paragraphs but from ongoing practice, study, and in-depth consultation with experts. Jun 9, 2014 at 20:57

If all your regressor variables are binary, then the linearity assumption is vacuous! so can be ignored. But you say you have variables like age, which is not binary. Then you can consider using a spline of age instead of age directly, which leads to GAMs (generalized additive models) or the use of regression splines. I found this useful when there are one or a few such variables.

• Hey sir, do you have a good resource for the mathematical side of GAMs? Jun 1, 2017 at 18:46
• Elements of Statistical Learning has a good section on GAMs, IIRC.
– Will
Jul 10, 2017 at 22:30

Here are some thoughts:

1. If you only want to identify relevant variables (e.g., you don't want to test pre-existing hypotheses, and you aren't trying to build the optimal model for predicting favorability later), you might try either CART, or combining logistic regression with LASSO penalization.
2. There is no assumption of 'linearity' in logistic regression. In general, people consider LR to be a non-linear model. There is some sense in which any model assumes it is properly specified, and it is possible (with a continuous variable) that you could need a squared term (e.g.) for the model to be reasonably well specified, but even that isn't true with only dichotomous variables.
3. It is often best to try to reduce your independent variables first. However, you still won't have to worry about 'linearity'.
• No, there IS an assumption of linearity. Wiev the logistic regression as a GLM (generalized kinear model). The linearity is in the linear predictor, while the probability itself is a non-linear function of that linear predictor. Else, what you says make sense. Jun 9, 2014 at 21:08
• gung: you are probably right it is more of a semantic reference! But, however it is expressed, there is an assumption there, which is void if the predictor is binary. Jun 9, 2014 at 21:17
• @kjetilbhalvorsen, that's certainly true. Jun 9, 2014 at 21:21
• @kjetilbhalvorsen and gung: Thank you both for your comments and advice! Does this mean that a LR model is invalid, or I don't need to worry about linearity? I think I will try Gung's suggestion of the LR with LASSO penalization and reduced number of IVs. To be honest, I have suggested that this analysis be much better viewed as a cluster analysis rather than deriving a formula for an "ideal" respondent - but need to show that we've attempted this approach nonetheless. Jun 9, 2014 at 22:19
• In this case (ie, your binary variables) you don't need to worry about linearity, @APCOInsight. Jun 9, 2014 at 22:27