0
$\begingroup$

I am trying to develop a model for prediction of retention. The problem is that the retention is very rare - aprox. 0.2 %. So far I have been using logistical regression. Without much success however. For example, in the interval of predicted probability above 70 % I am getting 4 true retention clients and 157 wrongly predicted retentions.

I have spent quite a lot of time deciding what covariates to choose and what transformation to apply. I don't think I can improve that.

The question could be formulated also as the following. How to use (or what to use instead of) logistical regression when the response vector "Y" contains very few 1's (0.2 % in my case)?

If you think you should know what data I have, tell me and I will provide you with this information. But I don't think it's important. Anyway, I have enough data. About 1.5 mil rows, so about 3000 of 1's.

$\endgroup$

marked as duplicate by Scortchi Oct 6 '15 at 12:47

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ (1) usually use the term "logistic regression" (2) what are your events per variable. if >>10 then you likely have enough data. if EPV <20 could consider regularization (3) logistic regression will provide best model unless there are significant interactions or non-linear effects your not modeling. (4) prediction is hard. what makes you think your model isn't good $\endgroup$ – charles Oct 6 '15 at 11:47
1
$\begingroup$

You have a quite high lift in the scored data. So your predictions are not necessarily "bad". What are classification statistics (e.q AUC) in the test data set?

$\endgroup$
  • 1
    $\begingroup$ (+1) Though a 2.5% observed retention vs over 70% predicted retention suggests some miscalibration. $\endgroup$ – Scortchi Oct 6 '15 at 12:54
  • $\begingroup$ I guess you are right. I will have to put more effort into the model. $\endgroup$ – T. Rubin Oct 8 '15 at 7:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.