I have fit two different multiclass classifiers on the same data, one is a multinomial logistic regression the other one a KNN Classifier. Now I want to know which one has a better fit in general.

I calculated the multiclass AUC given by Hand & Till (2001) for each one on the test set, which produced a value of ~0.6 for the logistic model and a value of ~0.53 for the KNN Classifier. Is there a way to test if that difference is significant?

The only literature I was able to find is concerned with AUC of binary classifiers.

Hand, D. J., & Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine learning, 45(2), 171-186.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.