Which quality measures are available for non-binary problems? I'm wondering which quality measures are available for non-binary classifiers. I've read this article
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
I understand, that the idea of Confusion matrix can be generalized easily. However, such metrics as sensitivity,specificity or precision seem to make sense only for binary classification problems. 
So, my question is: are there any developed methods for evaluting the quality of non-binary classifiers?
Thank you.
 A: I'm not sure whether you're still interested in this topic or not, but I became curious about it and have read a little on the subject. I've decided to share the following information to store the findings for myself and with hope that this might be helpful for others as well.
It seems that there are quite a number of quality measures, which can be applied to various models, non-binary classification included. Some of them are mentioned on this page and in this paper. Another interesting paper, criticizing AUC measure can be found here. Please note that my answer might be a little more general, as I haven't restricted my search to classification problems.
A: Kappa Statistic can be used. 
k = (Acc - AccRand) / (1 - AccRand)
k = 1 is ideal, k = 0 means that the classifier is not better than random.
Acc is the accuracy of the classifier measured as the number of correctly classified observations over the total number of observations. 
AccRand is the accuracy of a classifier making predictions by chance. AccRand can be easily measured empirically: take the outputs of your classifier, randomly permute predictions over observations, measure the accuracy.
