3
$\begingroup$

The area under the convex hull of a roc curve is by construction always "better" than its area under curve. Some curves might see more of an increase in reported auc than others. Is it a viable approach to prefer classifiers that "benefit more" from this approach in model selection?

A classifier that performs decent over the entire range of fpr-tpr thresholds could be discarded in favor of a classifier that excels in a specific region in the roc space and underperforms in others. The second classifier might be preferable regarding properties of its convex hull.

I am not sure whether to include this in a cross-validation process.

$\endgroup$
3
  • 2
    $\begingroup$ I think you touch upon something very important: The fact that AUC is not an all-encompassing truth but rather a single number. As you correctly say, if we know that a particular range of TPR or FPR is more relevant for an application then we should focus there. Similarly though we should recognise that if our ideal performance is unattainable (e.g. a classifier with 90%+ precision where we would have nonsensically low recall for a particular application), using AUC gives us a good indication what would be a generally useful classifier instead of one based on an arbitrary threshold. $\endgroup$
    – usεr11852
    Mar 14, 2020 at 0:13
  • $\begingroup$ I think that this answer can give you a better insight to your question. stats.stackexchange.com/q/132856 I feel that you want to know how to create the best classifier using a convex hull of roc curves between different models (?) $\endgroup$
    – M. Chris
    Mar 18, 2020 at 18:23
  • $\begingroup$ I know how to combine them - but due to properties of them convex hull, shouldn't the model selection change, since two "better classifiers" do not necessarily result in a better convex hull. $\endgroup$ Mar 19, 2020 at 6:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.