Concept Clarification: Logistic Regression Assumption

Question

Here is one of the assumptions under logistic regression: logistic regression assumes linearity of independent variables and log odds.

I understand if this assumption is violated, we can then transform (e.g. log, spline) the continuous predictor. However, my question is, what should I do if the assumption is still not met even after transformation?

Answer 1

This is an excellent question. Instead of transforming the binary response variable, another approach would be to gain insight into the individual relationships between the independent variables and the probability of one of the states in the binary response variable. You can gain insight in a variety of ways, and I will mention two ways:

1. plot the relationship between the independent variables and the empirical logits. The empirical logits are ln(phat/1-phat) calculated over bins/buckets of the independent variables. Please see the following for an example of how to plot these in R:

https://alexschell.github.io/emplogit.html

These plots will give some insight into transforming the independent variables to make the relationship linear. For example, you made need to add polynomial factors of the independent variables.

2. Understand the phenomenon you are modeling, make hypotheses about the relationships between the independent variables. Then you will be able to transform the independent variables and try models based on your assumptions.

For example, in a multiple regression or forecasting context, if I were modeling hourly electrical demand and temperature was an independent variable, I would certainly include a quadratic factor of temperature in the model since electrical demand goes up in the winter and summer due to the power required to run HVAC systems.

I hope this helps,

Best, =K=