• We use supervised learning algorithm to detect fraud, but we have fewer or even no positive samples, is it reasonable if we use rules to label positive samples? if so, is the supervised learning model is meaningful?
  • If we make use of anomaly detection to catch fraud, we use rules to label data in model evaluation phase, will the model work?
  • All above, if we can find rules to label data, does the model we build to detect fraud make any sense? because we've already used rules to find fraud.
  • 1
    $\begingroup$ What are the rules you are talking about? $\endgroup$ – Tim Apr 19 '17 at 7:57
  • $\begingroup$ Some rules based on the common sense, eg. to detect anomaly traffic, we define a rule that proxy ip addresses lead to anomaly traffic, and some statistic threshold rules. $\endgroup$ – fiona Apr 19 '17 at 8:53

If you have no positive cases in your data and instead you will use some expert rules to mark the cases you believe to be frauds, then obviously your model will learn how to classify the cases that were marked by you as frauds. There is no guarantee that it will learn anything at all about the true frauds, since it will simply be learning your rules. If you have data with no positive cases, then you should rather use methods designed for anomaly detection (see ).

  • $\begingroup$ As I mentioned in the question, if I use anomaly detection to detect fraud, such as Gaussian Model, we need to use labeled data on validation dataset in model evaluation phase to select features and tune parameters. Also the positive samples are labeled by some expert rules, will this model work? What influence it will have on this model? $\endgroup$ – fiona Apr 19 '17 at 9:44
  • $\begingroup$ The same as described above. Moreover, there are anomaly detection methods that are designed to work with only-negatives data! $\endgroup$ – Tim Apr 19 '17 at 9:47
  • $\begingroup$ As my data is heavy tail distribution, I choose Gaussian Model to detect fraud. Could you show me some anomaly detection methods that are designed to work with only negative data? I do not understand how to evaluate the model and tune the parameters without positive cases? $\endgroup$ – fiona Apr 19 '17 at 10:04
  • $\begingroup$ E.g. one-class SVM: rvlasveld.github.io/blog/2013/07/12/… $\endgroup$ – Tim Apr 19 '17 at 10:06
  • $\begingroup$ There do not only exit negative cases in my data set, which are positive and negative cases mixed, and extremely biased. The point is I do not know which record is anomaly, then which algorithm should I choose in this situation? $\endgroup$ – fiona Apr 19 '17 at 10:58

There is a very interesting work that was published in ICLR 2020:

Learning from Rules Generalizing Labeled Exemplars

This paper assumes that you have a bunch of rules and some labeled examples supporting each rule. It also assumes that you have access to a large unlabeled data. Then it proceeds to learn a model that can learn from rule exemplars and the unlabeled data jointly to do better than the rule sets themselves.

Given your setup, I felt that this work might be of some use to you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.