I am working on a classification problem with a highly imbalanced dataset. The ratio background to signal is about 20.
I trained an xgboost model. The ROC curve looks perfect and ROC_auc is also almost perfect 0.99. But the BDT response or probability to be[ a signal for a signal (training and test sets) looks very incorrectly.
I tried to balance the data by adding weight to the data, but without success. Can you give me any advices how to deal with highly imbalanced datasets?