1
$\begingroup$

I want to classify objects by their area into two classes. I implemented several area estimates that I want to compare. For each object, I have a gold standard indicating to which of the two classes each object belongs.

The area estimators give me a binary output, telling to which of the two classes the object belongs taking a threshold.

Now I want to compare these binary classifiers where each of them also applies several thresholds for the classification. So for every threshold, I can calculate the true positive rate and false positive rate and plot these values against each other. So I get one curve for each classifier and every datapoint stands for one threshold that is applied.

Reading through articles about ROC plots my impression is that in most cases they are used in a different context, namely the classifier outputs are not binary values but probablities to which a threshold is applied.

Are my TPR vs. FPR plots even ROC plots? Does my approach even make sense?

edit: the classifiers I want to compare and for which I want to compare thresholds estimate areas: area = [ 11.2, 20.3, 1.2, 3 ]; and apply a threshold to this area, resulting in the binary outputs outputs = [ 0, 0, 1, 1];

$\endgroup$
  • $\begingroup$ can you give a couple of rows of your training data (with input and output variables) to clarify. $\endgroup$ – Zhubarb Jun 19 '15 at 7:34
2
$\begingroup$

Yes, this sounds like a correct ROC curve. It is true that often these are created from probability estimates, but there's nothing that requires that be the case; arbitrary thresholds are fine too.

$\endgroup$
  • 1
    $\begingroup$ A common example of non-probabilistic scores is the (signed) distance to the separating hyperplane for SVMs, for which scores are in $\mathbb{R}$. $\endgroup$ – Marc Claesen Jun 19 '15 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.