10
$\begingroup$

Especially in the computer-science oriented side of the machine learning literature, AUC (area under the receiver operator characteristic curve) is a popular criterion for evaluating classifiers. What are the justifications for using the AUC? E.g. is there a particular loss function for which the optimal decision is the classifier with the best AUC?

$\endgroup$
  • 1
    $\begingroup$ AUC is a loss function, it is clear that for this loss function the optimal decision is the classifier with the best AUC. $\endgroup$ – robin girard Jun 5 '11 at 19:23
  • 1
    $\begingroup$ @robingirard No it's not, since it's not differentiable, i.e. you can't optimize it directly. $\endgroup$ – cpury Jun 22 '18 at 15:20
15
$\begingroup$

For binary classifiers $C$ used for ranking (i.e. for each example $e$ we have $C(e)$ in the interval $[0, 1]$) from which the AUC is measured the AUC is equivalent to the probability that $C(e_1) > C(e_0)$ where $e_1$ is a true positive example and $e_0$ is a true negative example. Thus, choosing a model with the maximal AUC minimizes the probability that $C(e_0) \geq C(e_1)$. That is, minimizes the loss of ranking a true negative at least as large as a true positive.

$\endgroup$
1
$\begingroup$

Let's take a simple example of identifying good tomato from a pool of good + bad tomato. Let's say number of good tomato are 100, and bad tomato are 1000, So a total of 1100. Now your job is to identify as many good tomato as possible. One way to get all good tomato are taking all 1100 tomato. But it clearly says you are not able to differentiate b/n good vs bad.

So, What's the right way of differentiating - need to get as many good ones while picking up very few bad ones, So we need a measure something, which can say how many good ones we picked up and also say what's the bad ones count in it. AUC measure gives more weight if it's able to select more good ones with few bad ones as depicted below. which says how good you are able to differentiate b/n good and bad.

In the example you can observe that while picking up 70% good tomato, black curve picked up around 48% of bad ones (impurity), but blue one has 83% bad ones (impurity). So black curve has better AUC score compared to blue one. enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.