1
$\begingroup$

I am doing some predictive churn modelling. I use around 250 independent variables (not all at same time). Those variables are transactional, demographics, external data etc. The database is fairly large, 100' plus. I am using backward elimination with 0.01 as P-value lever to enter model.

After I have run the logistic regression and eliminated variables for correlation between variables, I have approx 35 significant variables.

My impression is that this is too many predictors and I would prefer too have less. Anyone that have an opinion of how many significant variables you can put in a model without overfitting and making it to complex?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Check this thread to learn more on pitfalls of stepwise selection stats.stackexchange.com/questions/20836/… - p-value criteria is actually one of the worst choices for selecting variables... $\endgroup$
    – Tim
    Commented Sep 10, 2015 at 10:20

2 Answers 2

Reset to default
2
$\begingroup$

You may want to apply LASSO/Elastic net approaches, as described in http://web.stanford.edu/~hastie/TALKS/enet_talk.pdf, per example. Tuning the penalty parameter, you can select the number of predictors present in the final model.

However, the best thing to do to evaluate the optimal lambda (in terms of prediction accuracy of your model) is to run a k-fold cross-validation and chose the value that minimized your error, not setting it to have 10 or 20 predictors in the end.

Improve this answer
$\endgroup$
1
$\begingroup$

There is no golden rule to answer your question. Since this is application specific the best way might be to divide your data into a training and a test sample. Then you can try out different models based on your training set and compare how well they fit your test data. Though 35 is indeed a high # of predictors...

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is a predictor count of 35 really that high for 100000+ observations? $\endgroup$
    – JonB
    Commented Sep 10, 2015 at 10:24
  • 2
    $\begingroup$ Depends on what what you intend to do with your model. As I mentioned before it really depends on the specific application and the data itself. $\endgroup$
    – Martin Schmelzer
    Commented Sep 10, 2015 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.