I am reading in different papers that it is impossible to optimize AUC in term of gradient descent, due to its non differentiability. I can't find any reference about this issue with a better explication and I can't understand why we can't compute AUC through mini batches. Who can explain me a little bit more?
Reproducing comments as an answer:
From Alex R.:
For starters, using AUC (ranking-based) loss functions results in a non-convex optimization, with plenty of local maxima. Usually people re-transform the loss function with a surrogate that is convex. See here for example: On the Consistency of AUC Pairwise Optimization Wei Gao and Zhi-Hua Zhou
For another thing, the optimum wouldn't be unique. All that would matter is the rank ordering. For example, estimating logistic regression coefficients, a logit probability of $.01+.01x$ would produce the same AUC as $1000+1000x$. With a single predictor, all that would be identified is the sign of the coefficient. With multiple predictors, their sizes relative to each other might be identified, but there would be no unique solution