The question is, "How to select predictor variables for a classification model?"
In order to give a specific answer, more information would eventually be needed about your dataset and specific application.
In churn analysis, the goal for a predictive model is achieve the highest prediction score possible. Variable selection helps to create simpler models, but can increase or decrease performance.
Variable selection occurs in multiple stages:
- Variable selection in the data pre-processing stage
- Variable selection in the model building stage
"Variable selection" includes the process of removing variables that fail to meet the assumptions of your model (i.e. non-normality, imbalanced classes, multicollinearity) and you may have to remove some variables or gather more data.
- Multicollinearity can lead to inflated variable importance
- Too many variables for too few samples can be overfitting
- Imbalanced classes can lead to poor performance
If you have problems such as these in your data, then variable selection may produce drastically differing results each time you train the model.
Once all your model assumptions are met, here are some techniques that may help. Keep in mind that each has its own benefits and drawbacks depending on your particular application and prediction model.
- Try adding new variables using data from outside sources (i.e. weather, city data)
- Try variable transformations
- Try creating new features from existing data
- Try higher order prediction models
There are many other approaches to this problem, hopefully this will give insight to some people :)