Churn prediction on a highly passive and imbalance dataset

Question

I'm trying to create a model to predict churn in the insurance industry. The objective will be to ' predict the probability of each member that will churn next month' i created a one row per member per month dataset where every member has one row for every month he has been active, the demographical information during that month, the household info, the claims made that month, the premiums paid and the calls made to the customer contact center. Below is the sample of what the dataset might look like

Id Year Month product.x product.y claim prem calls

x 2017 8 1 0 250 180 1

x 2017 9 1 0 270 180 0

x 2017 10 1 0 0 180 0

x 2017 11 1 0 250 180 0

Now to the question: I'm trying to predict who is going to churn next month. the test dataset is the active members at the end of this month. the training dataset is the rest of the data, which also has the same members + few members that churned. Since the members in the test dataset are also present in train dataset without any variation across months( in the insurance industry, there is no major change in the activity of individual across months. they pay the same premium as last month, most buy regular drugs-hence regular claims and the demographics change rarely), the model always results that all members will be active.

How do I go about solving this issue? Am I making a mistake in the split of dataset?

Any thoughts will be highly appreciated

update for mkl:

you're right. the data is far from perfect. but we are getting there by incorporating more variables. Currently, I tried the following: • Performed k fold cross-validation on the entire dataset with tree ensemble learner and random forest (predicts all customers to be active. Zero kappa score) • I made a split on the one row per member per month dataset in such a way that I have a designated test dataset consisting of all the members that are active at the beginning of the month. The remaining is the training dataset. So, the training dataset contains the termed members and each member has multiple rows for every month till they terminated. Now the training dataset is quite small (the terminated members in a year are very few compared to the active members in the training dataset). Did oversampling of train data and used GBM. Still no results

Answer 1

Don't predict yes/no. Instead, predict a probability that someone will churn. Then base any actions (e.g., calling them, offering a discount, whatever) on that probability and the costs of correct/wrong actions.

This earlier thread may be helpful.

Answer 2

The questions is whether the data you collected can explain churn at all. The model can learn some weights / thresholds but it will not extract knowledge / rules that are outside your data. E.g. if the most influential factor is (change of) income and you don't have it in your data, your predictions will be far from perfect.

Assuming positive answer - i.e. that observing your features can give you a reasonable answer about probability of the churn - how do you predict the churn? What is your model / approach?

Do you use some aggregates over the features and their combinations, like 'sum of claims > sum of premiums over last 6 months', 'number of calls is increasing in every month through last 3 months'?

Answer 3

Why don't you assign observations to Train and Test based on ID?

create a column that is ID mod 10, then use if/then logic to assign. Then you won't have the same policyholder in both data sets.

I would also add I would define a fold column based on ID. You've likely got the same person in all of your folds.

Is this insurance bought through an employer or is it direct? Are there variations by time of year?