I am trying to understand the interpretation of these metrics in a multiclass scenario:
MAUC. Scikit-learn provides an implementation for ROC-AUC score, which can be used for both binary and
However, some studies such as 2, 3, 4 and 5 suggest averaging class-wise AUC in
My experiments with this metrics yield different results. Do they then evaluate to different quantities?
For clarity, I used 4 implementation as well as sklearn's ROC-AUC. I the case of
sklearn's, I set the hyperparameters as:
roc_auc = metrics.roc_auc_score(y_test, ypred, average='weighted', multi_class='ovo',labels=labels)
Random Forest classifier, we obtained:
ROC-AUC: 0.58 # sklearn's roc-auc-score MAUC: 0.69
This is more than a
10% difference so the two values are not close at all.
1 sklearn's roc-auc score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html
2 Tanha, J., Abdi, Y., Samadi, N., Razzaghi, N., Asadpour, M.: Boosting methods for multi-class imbalanced data classification: an experimental review. Journal of Big Data 7(1), 1–47 (2020)
3 Wang, R., Tang, K.: An empirical study of MAUC in multi-class problems with uncertain cost matrices. CoRR abs/1209.1800 (2012), http://arxiv.org/abs/1209.1800