2
$\begingroup$

I have a binary classification problem. The task is to rank instances from high probability of fraud to low probability of fraud.

The following data is available:

~7.000 instances of 0/1 labeled data, roughly 20% 1's, 80% 0's. The labels were obtained in part of the population for which fraud was already suspected, so this part of the population is NOT representative for the entire population. In particular there are several subgroups of the population that were heavily over/under inspected in the past.

~55.000 instances of unlabaled data. Expected rate of y=1 in this part of the population is somewhere between 1-10%

We currently do the following:

-Assume all the unlabeled instances have in fact label y=0. Apply a machine learning technique on all instances. So y=1 for all instances that we know for sure that y=1, and y=0 for all the rest.

After various analysis we strongly suspect the above procedure to lead to better predictions than if we would use the standard procedure of only using labeled data.

However, there might be ways such that we can utilize all available information even better?

Ive been looking into semi-supervised learning literature but I dont know if methods proposed there will work in case the labeled data is biased. Furthermore it is difficult to compare methodology because there is no unbiased test set.

$\endgroup$

1 Answer 1

3
$\begingroup$

This is called PU learning (Positive and Unlabeled learning). Several approaches exist to account for this situation, for instance my own recent publication (post-print available here).

As you mention, ignoring the unlabeled data entirely is a bad idea. Most existing approaches (cfr. my paper and references therein) treat the unlabeled set as if it were negative while accounting for the decreased lack of confidence in those labels.

The fact your labeled data stems from a biased sample will inevitably affect the resulting model. Evaluating the resulting model's performance will be particularly challenging.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.