# imbalanced dataset

I am working on a classification problem with a highly imbalanced dataset. The ratio background to signal is about 20.

I trained an xgboost model. The ROC curve looks perfect and ROC_auc is also almost perfect 0.99. But the BDT response or probability to be[ a signal for a signal (training and test sets) looks very incorrectly.

I tried to balance the data by adding weight to the data, but without success. Can you give me any advices how to deal with highly imbalanced datasets?

• What is the cost function you train the classifier with? How do you pick the threshold for the actual classification? What is BDT? – usεr11852 says Reinstate Monic Apr 28 '17 at 20:43

imbalanced-learn is a Python library that implements some of the aforementioned algorithms, has a simple interface and it is compatible to scikit-learn.