0
$\begingroup$

So I'm using survival analysis to help identify customers at risk of churning. As of now, I've been using a hugely imbalanced dataset that consists of 99.5% of customers in the majority class (i.e. not churned) and 0.5% churned. To improve my model I've randomly undersampled the majority class such that the ratio between the minority and majority classes is only 1/3, which resulted in improvements when compared to using the full dataset.

However, in order to avoid removing data points which can be important, I would like to try oversampling techniques such as SMOTE, etc but I don't know how well this would suit survival models, because the duration (target variable) depends on a churn date/time which is the result of the difference of two variables in the dataset (date_client_churned + date_client_joined). Isn't generating new examples going to mess with the distribution of these two variables and as such produce erratic results? Has anybody dealt with this kind of imbalance in the dataset for survival analysis, and if so what was your approach? Is it really necessary to balance the dataset for survival analysis?

For reference, the survival models that I've used are: Cox Proportional Hazards, Random Survival Forests and Gradient Boosted Trees with Cox loss.

Just to clarify how I compute the duration variable: For time t = 0 I'm using the date a customer signs a contract with the company, given by t_born, and for the end date I'm using either a) the churn date t_churn if t_churn < t_study, where t_study represents the end of the study period (t_study=t_born+6, t_study=t_born+12, t_study=t_born+24 months); b) t_study otherwise. The duration column will then be given by the difference t_churn - t_born if t_churn < t_study; t_study - t_born otherwise.

$\endgroup$
10
  • $\begingroup$ It seems at first glance that the duration should be the difference of two dates, date_client_churned - date_client_joined, to represent the length of time that the individual was a client. Please edit the question to explain what you are using for the reference time = 0 for individual customers and why you are using the sum of those two dates for the duration survival time. Please do that by editing the question, as comments are easy to overlook and can be deleted. Also, note that this might better be handled by a cure model that accounts for individuals that never have events. $\endgroup$
    – EdM
    Aug 9, 2022 at 15:00
  • 2
    $\begingroup$ Misconceptions abound when it comes to class imbalance, and statistics sees little problem. Why do you think the class imbalance is problematic in your case? $\endgroup$
    – Dave
    Aug 9, 2022 at 15:32
  • $\begingroup$ The fact that you would create a new topic when this issue has been covered extensive on the site. Thinking class imbalance is a problem is equivalent to not understanding the field of statistics. $\endgroup$ Aug 9, 2022 at 15:58
  • $\begingroup$ @EdM Thank you for the suggestion, I've added an explanation on the OP. $\endgroup$
    – tomas_s
    Aug 9, 2022 at 16:34
  • 1
    $\begingroup$ Any apparent improvement that comes from disrespecting the sample sizes is an illusion. And don't use the c-index to compare models. It's not good at that. $\endgroup$ Aug 9, 2022 at 23:17

1 Answer 1

1
$\begingroup$

I would like to try oversampling techniques such as SMOTE, etc but I don't know how well this would suit survival models...

As the comments on this question and many pages like this one demonstrate on this site, it's not clear that SMOTE or any type of biased over/under-sampling suits any type of model well. You then necessarily bias the intercept of the model and thus all probability estimates (unless you back-correct for the biased resampling). It's not clear that even relative log-odds or log-hazard estimates are improved, as you don't have any more cases than you started with.

Unbiased resampling by bootstrapping at the original sample size, however, is a very useful technique for validating and calibrating models, including survival models. See the validate() and calibrate() functions of Frank Harrell's rms package for example. Provided that you resample by case so that each set of t_born, t_churn and t_study is kept together, there is no problem.

I suspect that your apparent improvement with SMOTE would disappear if you evaluated performance of the modeling process by bootstrap resampling or (with tens of thousands of cases) repeated train/test splits.

Finally, it's not clear that a continuous-time survival model like a Cox model is the best choice for data like yours, where there is apparently a very large ratio of cases to observation times. A discrete-time model--a set of binary regressions, including time as a predictor, on appropriately structured data--could be preferable. In particular, it would naturally handle your large proportion of censored cases that might be considered "cured" in a survival model. At the least you should consider a cure model that combines survival over time with a binary "cured" outcome, as the types of model you have used implicitly assume that every case eventually experiences the event in question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.