I am trying to predict customer churn in a telco company, using R.The dataset is very unbalanced, the target is around 0.6% of the base.

  • 8,746 Customers will Churn
  • 1,396,664 Customers do not churn

I have trained a Random Forest in R.Prior to training, I SMOTE the training data:

train.smote <- SMOTE(Churn~ ., train, perc.over = 100, perc.under=200

This gives me a 1:1 Balance. I then train the forest using:

fit<- randomForest(as.factor(Churn)~.,data=train.smote,importance=TRUE, 

When I run,


on my validation Data, I get the following Confusion Matrix:

             Positive    Negative
Positive     1,136,610   234,625 
Negative     3,762       5,911 

The F Score is 0.83, the Specificity is 0.61. However, the number of False Positives is too high (234,625). Please suggest a method to curb these False Positives without compromising on the True Positives.


Probably this question should be moved over to CrossValidated or DataScience.

Anyway, there are tons of things you can do.

Feature Extraction and Importance, which variables are most correlated with the churn, which are less, remove the latter.

Can you extract new feature from the variables you have? This can be hard and a bit of "try and fail", but it can improve your results.

At last, try Ensemble Models, essentially you build several different models (best if uncorrelated with each other), and combine the results. This will boost the performance. Ensemble Models in R.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy