Splitting between train/test for customer churn survival models

Question

I am a bit confused on how data can be split between train/test and "live" data for predicting churn using survival models such as the one in RandomForestSRC package.

Goal of the model is to predict how long a currently active customer will remain a customer before churning.

The current dataset includes customers who have already churned and customers who are active. Say we have 1000 customers who have already churned and 1500 active customers.

My conundrum is that these 1500 active customers (right censored) would be part of the training data set for a survival model. However this dataset also represents the "live" dataset on which the model should be applied to predict when they might churn. That seems wrong because now I end up using the same data to train and then to predict.

Any thoughts on how data can be split for train/test/"live" predict?

E.g. of data set I am using: http://www.sgi.com/tech/mlc/db/churn.data Names - http://www.sgi.com/tech/mlc/db/churn.names

Answer 1

I think this depends on whether you have a cross-section (an "image" of the dataset taken at a particular time) or you have panel data.

If it's just a cross-section, then just select observations randomly. The training sample will have both people that stay and people that leave, and so will the test sample. No one would appear in both. If the churn population is very small, you might consider oversampling them.

If you have panel data and you observe people repeatedly in time, you can consider setting your training data up to a certain date and the test data the remaining months. You'll have to make sure to leave enough months so that the test data has enough people that leave. In this case you'd have people in both training and test data. I guess the only way this makes sense is if some individual features vary over time (even if it's just something mechanical like "time elapsed" since subscribing!).