I am facing a problem with the classification of an imbalanced sample, and I was looking into solutions for both dealing with the imbalance (e.g. oversampling/undersampling) and a metric which would be suitable for the estimation of the classifier performance on the imbalanced sample.

I need to mention that my sample is not extremely imbalanced, at most 1:2 to 1:3 ratio.

Regarding the metrics, I found many alternatives to accuracy, except for the one which seemed very logical to me -- what I call in my head "normalized accuracy". Simply said, it is the average of the ratios of the correctly identified samples within each class. On a confusion matrix for two classes, it would be (TP/(TP+FN) + TN/(TN+FP))/2. This seems very logical to me, symmetrical and extendible to multilabel classification.

enter image description here

However, when I tried to find any mentions in the literature regarding this, I completely and repeatedly failed. Is it known under another name? Is it somehow so flawed that nobody ever discusses it anymore? Can someone enlighten me about this?


Your measure is a linear transform of Youden’s J statistic. See wikipedia Youden's J statistic page for more information about how is computed. J statistic has one advantage, it’s range which make it easier to compare.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.