I'm modeling credit fraud, where I have a small number of samples that result in fraud (1), and most samples that are not fraud (0). I am creating a models for detecting fraud based on new data.
I'm using the following models: logistic regression, K-nn, Support Vector Classifier and decision tree. The dataset is very similar to this: kaggle.com/mlg-ulb/creditcardfraud
I performed random undersampling on the data to get a 1:1 ratio. This made my models perform a lot better, but since the undersampling is performed randomly every time, I get a slightly different result because of the chosen samples.
Is there a way to find out which of the 8200 majority class samples are best to use in the undersampled data?
I was hoping to figure out which these are and only use them with the 800 positive samples on my undersampled dataset.