I have fit two different multiclass classifiers on the same data, one is a multinomial logistic regression the other one a KNN Classifier. Now I want to know which one has a better fit in general.
I calculated the multiclass AUC given by Hand & Till (2001) for each one on the test set, which produced a value of ~0.6 for the logistic model and a value of ~0.53 for the KNN Classifier. Is there a way to test if that difference is significant?
The only literature I was able to find is concerned with AUC of binary classifiers.
Hand, D. J., & Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine learning, 45(2), 171-186.