In fraud investigation the number of detected fraud cases can be very small when compared to the total number of cases. This would also apply to rare desease detected in a very small number of people in the population. In these cases we have a limited number of investigated cases and from these cases we know the target (fraud cases or patients with the desease) and non-target (not fraud or patients without the desease) cases.
Since in these cases the target observations only account for, let say, 1% of the total number of cases we will have a very imbalanced data and most classification models will classify as non-target almost all the time, so it will not be very useful. One way to tackle this kind of problem is to use only the investigated cases in the training dataset so that we will have a less imbalanced data. The problem with this approach is that the distribution of the data used to build the model will be different from the data that it will make predictions on. In this case we deliberately have a selection bias since we are just using investigated cases, however there are ways to address this issue and obtain better results.
I have been searching for a technique to deal with this problem but there are few studies available. The best one I found so far is the following: On Selection Bias with Imbalanced Classes. This paper says that we should use a subsample of the unlabled data, that is the cases that were not investigated, and consider them as non-targets. However, in the paper I could not find exactly how to subsample the unlabled data (how many, which ones). I would like to know if anyone knows the steps to take in this kind of situation or any other techniques to address this issue.