1
$\begingroup$

I have to use a roc curve to find an agreement between sensitivity between the FPR and TPR. The problem is that I only know the true positives. How could I circumvent this? More accurate, I have a vector with some points known to be true, but I have no idea about the other points. How could I get a ROC curve?

Thanks

$\endgroup$
  • 1
    $\begingroup$ I am actually about to submit a manuscript to this year's ICML that address this exact problem. I will update you when it is available. In case you need an urgent fix, feel free to contact me directly. Essentially you only need a set of known positives and a rough estimate of how many positives you have in the unlabeled part of your test set to get very reliable estimates of the corresponding ROC curve. $\endgroup$ – Marc Claesen Jan 30 '15 at 13:53
  • $\begingroup$ Hi Marc, Thanks for your answer. The problem is that I do not know that 'rough estimate' of the number of positives in the unlabeled part. However, I would like to hear more about your method, it sounds interesting to my project. $\endgroup$ – Peter Pfand Jan 31 '15 at 17:04
  • 1
    $\begingroup$ I'll update you in about one week (after submission). $\endgroup$ – Marc Claesen Feb 1 '15 at 11:32
  • $\begingroup$ Slightly later than I hoped, but here's the relevant paper Assessing binary classifiers using only positive and unlabeled data. $\endgroup$ – Marc Claesen Apr 28 '15 at 5:08
1
$\begingroup$

I don't think that you can. A ROC curve assumes that you have a binary classifier (the "ground truth" or "gold standard") and a predictor for each subject. If you don't have the binary classifier then you cannot form the ROC curve at all.

You should look at your study and see if you can use a modified classifier. For instance, I was doing an ROC analysis once for a test in the medical field. On some subjects we had pathology as our classifier, but on others we used clinical follow-up. I.e. if the suspected disease had not progressed at the 12 mo follow-up then we called it "negative" even though, in principle, they could have had very very slowly progressing disease.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.