Logistic Regression - Dummy and Numeric variables together I am trying to build a logistic regression model. I have some categorical variables for which I have created dummy variables (eg. Department). I also have some numeric variables like Age and Tenure. 
My question is which of the approach should I use- 


*

*Should I use a combination of dummy variables and numeric variables
as an input to my logistic model. 

*Or, should I create categories of numeric variables based on response rate and use these categories to create dummy variables for numeric variables as well.


In first approach I am afraid that I numeric variable will become highly significant and cause overfitting. Also, they will reduce the real "significance" of dummy variables. 
In second approach I am afraid that I will loose a lot of information.     
 A: Adding a numberic variable to a logistic regression is unlikely to lead to overfitting as it imposes quite a strong contraint: every unit increase in your numeric variable leads to an increase or decrease in the odds of success of a factor $\exp(\beta)$. By default this factor is constant, which is how you can describe that effect with just one number. You can relax that assumption by adding polynomials, splines, or breaking your numeric variable up into different categories. Overfitting starts to become an issue if you use a polynomial of too high order, too many knots or break your variable up in too many classes. 
So if anything your strategy 2 is in danger of loosing too much information if you choose too few categories or overfitting if you choose too many categories.
A: I suggest that you take the first approach. With the second approach, not only you might lose some information, but you can also introduce biases in the dataset by categorizing numerical features.
To make the first approach work, you need to pre-process all the features once you create dummy variables, which includes:


*

*Centering and scaling

*Transformation to remove skewness in data

*Remove highly correlated features

*Remove features that have near-zero variance 


To prevent overfitting, you can use penalized logistic regression model.
