4
$\begingroup$

I am reading in different papers that it is impossible to optimize AUC in term of gradient descent, due to its non differentiability. I can't find any reference about this issue with a better explication and I can't understand why we can't compute AUC through mini batches. Who can explain me a little bit more?

$\endgroup$
2
  • 1
    $\begingroup$ For starters, using AUC (ranking-based) loss functions results in a non-convex optimization, with plenty of local maxima. Usually people re-transform the loss function with a surrogate that is convex.See here for example: arxiv.org/pdf/1208.0645v4.pdf $\endgroup$ – Alex R. Feb 1 '17 at 18:52
  • 3
    $\begingroup$ For another thing, the optimum wouldn't be unique. All that would matter is the rank ordering. For example, estimating logistic regression coefficients, a logit probability of $.01+.01x$would produce the same AUC as $1000+1000x$. With a single predictor, all that would be identified is the sign of the coefficient. With multiple predictors, their sizes relative to each other might be identified, but there would be no unique solution $\endgroup$ – not_bonferroni Feb 1 '17 at 20:52
3
$\begingroup$

Reproducing comments as an answer:

From Alex R.:

For starters, using AUC (ranking-based) loss functions results in a non-convex optimization, with plenty of local maxima. Usually people re-transform the loss function with a surrogate that is convex. See here for example: On the Consistency of AUC Pairwise Optimization Wei Gao and Zhi-Hua Zhou

From not_bonferroni:

For another thing, the optimum wouldn't be unique. All that would matter is the rank ordering. For example, estimating logistic regression coefficients, a logit probability of $.01+.01x$ would produce the same AUC as $1000+1000x$. With a single predictor, all that would be identified is the sign of the coefficient. With multiple predictors, their sizes relative to each other might be identified, but there would be no unique solution

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.