I am working on developing a logistic regression model that uses qualitative variables only ($n=990$). My remit is to define the equation that can identify the most relevant characteristics of a survey respondent that is favorable towards Company X. The proposed equation is similar to the following: \begin{align} {\rm Fav} = &a*{\rm age} + b*{\rm CoAware} + c*{\rm IssueAware} + d*{\rm readnewspaper} + e*{\rm region} + \\ &f*{\rm income}\ldots \end{align} The dependent variable is "Company Favorability" (0 = Unfavorable/Neither | 1 = Favorable). There are currently 25 independent variables, 20 of which are binary IVs that range from highly correlated to the DV (awareness of Company) to not significant (gender). I also have 5 categorical variables that indicate region of the country, age (in categories), party affiliation, income level, and education.

I am almost certain that I need to use a logistic regression model for this approach. However, when I test my assumptions, I have having a very difficult time proving a linear relationship between the dichotomous independent variables and the logit transformation of the DV.

My other problem is that, I am somewhat overwhelmed by possible interaction effects. There are 34 possible options using 25 variables - leading me to over 50 million possible combinations.

I have three questions:

  1. Is there a better method to model with a binary dependent variable?
  2. Am I missing something in the assumptions? (ie: Do I need to indeed prove the linear relationship if all of my variables are dichotomous)
  3. Would it be better to approach this by looking at multicollinearity first, to reduce the number of variables overall, and then look at linear relationships with the logit of the DV?
  • $\begingroup$ I admire the work and the hard thinking that this question reflects (+1). You're concerned with many important issues with which many of us struggle. Unfortunately I think the answers will come not from a few paragraphs but from ongoing practice, study, and in-depth consultation with experts. $\endgroup$ – rolando2 Jun 9 '14 at 20:57

If all your regressor variables are binary, then the linearity assumption is vacuous! so can be ignored. But you say you have variables like age, which is not binary. Then you can consider using a spline of age instead of age directly, which leads to GAMs (generalized additive models) or the use of regression splines. I found this useful when there are one or a few such variables.

Then consider which interactions you consider plausible, then start with your model.

  • 1
    $\begingroup$ Hey sir, do you have a good resource for the mathematical side of GAMs? $\endgroup$ – Guilherme Marthe Jun 1 '17 at 18:46
  • 1
    $\begingroup$ Elements of Statistical Learning has a good section on GAMs, IIRC. $\endgroup$ – Will Jul 10 '17 at 22:30

Here are some thoughts:

  1. If you only want to identify relevant variables (e.g., you don't want to test pre-existing hypotheses, and you aren't trying to build the optimal model for predicting favorability later), you might try either CART, or combining logistic regression with LASSO penalization.
  2. There is no assumption of 'linearity' in logistic regression. In general, people consider LR to be a non-linear model. There is some sense in which any model assumes it is properly specified, and it is possible (with a continuous variable) that you could need a squared term (e.g.) for the model to be reasonably well specified, but even that isn't true with only dichotomous variables.
  3. It is often best to try to reduce your independent variables first. However, you still won't have to worry about 'linearity'.
  • $\begingroup$ No, there IS an assumption of linearity. Wiev the logistic regression as a GLM (generalized kinear model). The linearity is in the linear predictor, while the probability itself is a non-linear function of that linear predictor. Else, what you says make sense. $\endgroup$ – kjetil b halvorsen Jun 9 '14 at 21:08
  • $\begingroup$ @kjetilbhalvorsen, the "non-linear model" part is just a throwaway comment, you're right about that. But neither OLS nor LR really assumes linearity, there is simply the assumption that the model is properly specified, which may amount to linearity in some cases. I suspect this is more of a semantic difference. $\endgroup$ – gung Jun 9 '14 at 21:15
  • 1
    $\begingroup$ gung: you are probably right it is more of a semantic reference! But, however it is expressed, there is an assumption there, which is void if the predictor is binary. $\endgroup$ – kjetil b halvorsen Jun 9 '14 at 21:17
  • $\begingroup$ @kjetilbhalvorsen, that's certainly true. $\endgroup$ – gung Jun 9 '14 at 21:21
  • 2
    $\begingroup$ In this case (ie, your binary variables) you don't need to worry about linearity, @APCOInsight. $\endgroup$ – gung Jun 9 '14 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.