I was comparing the performance of various models on a binary classification task I have been working on. When I plot the ROC curve for Gradient Boosting, this is what I get:
However, when I apply a random forest model to the same data, I get an incomplete ROC curve, or certainly a very fishy one.
The Random Forest model's performance just seems to good to be true, is it really the case it is doing that well? When I check the probabilities, a lot of them have the value 1. I suspect an information leak, but then not sure how pandas' train_test_split could be causing that.