My data set is imbalanced - 5% of the target class represents fraudulent transactions, 95% of the target class represents legitimate transactions. I must use the whole data set, as the 95% of legitimate transactions are important for training. How can I use undersampling within algorithms such as rpart (decision tree), naive bayes, neural networks, SVM, etc. to create, run and evaluate using multiple splits of the data. For example: the number of legitimate transactions is equal to the number of fraudulent. So 5% and 5%. This is instead of the typical way of cutting down the data set to 50% legitimate, 50% fraud where you would lose 85% of the legitimate transactions. I cannot oversample by generating randomized data in this case.
First off, 95%-5% is NOT an example of imbalanced dataset, I would consider downsampling if there was something of the order of 99.9%-0.1%. Most approaches should work just fine.
Edit: Seems like this answer has this covered already, have a look here: https://stats.stackexchange.com/a/133385/41367;
Now, if you have to rebalance, here are some options:
- Weight the infrequent class
- Stratified Sampling to downsize the frequent class
- Balanced subsampling to create multiple random balanced subsamples, then bootstrapping, ensembling (eg. Random Forests) etc.
From what I interpret, 3rd point should address your question about undersampling within the algorithm.