I am working on a classification problem with a highly imbalanced dataset. The ratio background to signal is about 20.

I trained an xgboost model. The ROC curve looks perfect and ROC_auc is also almost perfect 0.99. But the BDT response or probability to be[ a signal for a signal (training and test sets) looks very incorrectly.

enter image description here

I tried to balance the data by adding weight to the data, but without success. Can you give me any advices how to deal with highly imbalanced datasets?

  • $\begingroup$ What is the cost function you train the classifier with? How do you pick the threshold for the actual classification? What is BDT? $\endgroup$ – usεr11852 Apr 28 '17 at 20:43

Learning under imbalanced datasets is an active research area in machine learning [1]. Algorithms that have to deal with imbalanced datasets can be categorized into 4 groups: (i) under-sampling, (ii) over-sampling, (iii) combination of over- and under-sampling, and (iv) ensemble learning methods.

imbalanced-learn is a Python library that implements some of the aforementioned algorithms, has a simple interface and it is compatible to scikit-learn.

[1] http://ieeexplore.ieee.org/document/5128907/

| cite | improve this answer | |
  • $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? We can also turn it into a comment. $\endgroup$ – gung - Reinstate Monica Dec 20 '16 at 19:07
  • $\begingroup$ @gung I extended my answer $\endgroup$ – chkoar Dec 23 '16 at 7:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.