# Too many False Positives with Unbalanced Data

I am trying to predict customer churn in a telco company, using R.The dataset is very unbalanced, the target is around 0.6% of the base.

• 8,746 Customers will Churn
• 1,396,664 Customers do not churn

I have trained a Random Forest in R.Prior to training, I SMOTE the training data:

train.smote <- SMOTE(Churn~ ., train, perc.over = 100, perc.under=200


This gives me a 1:1 Balance. I then train the forest using:

fit<- randomForest(as.factor(Churn)~.,data=train.smote,importance=TRUE,
ntree=500)


When I run,

pred=predict(fit,newdata=test,type="class")


on my validation Data, I get the following Confusion Matrix:

             Positive    Negative
Positive     1,136,610   234,625
Negative     3,762       5,911


The F Score is 0.83, the Specificity is 0.61. However, the number of False Positives is too high (234,625). Please suggest a method to curb these False Positives without compromising on the True Positives.

## migrated from stackoverflow.comAug 21 '18 at 13:20

This question came from our site for professional and enthusiast programmers.

## 1 Answer

Probably this question should be moved over to CrossValidated or DataScience.

Anyway, there are tons of things you can do.

Feature Extraction and Importance, which variables are most correlated with the churn, which are less, remove the latter.

Can you extract new feature from the variables you have? This can be hard and a bit of "try and fail", but it can improve your results.

At last, try Ensemble Models, essentially you build several different models (best if uncorrelated with each other), and combine the results. This will boost the performance. Ensemble Models in R.