2
$\begingroup$

I am working on my multi-class classification project and I have a question:

I have three classes in proportion: 50%, 47% and 3%. I decided to use class_weight="balanced" parameter in random forest classifier. Now I want to calculate accuracy. Should I use balanced accuracy or can I use common accuracy?

$\endgroup$

2 Answers 2

6
$\begingroup$

Against @gunes, I defend that you can use whatever metric you want. Yes accuracy may give you unexpected results in a imbalanced problem, but the choice of metric is based on the needs of your problem. If all classes have the same cost for errors, then accuracy is the correct metric. What is usually implicit is that the cost of missing a low frequency class (in your case the class with 3%) is higher than missing the other classes. But it may not be so. Maybe your problem is directing software tickets to the appropriate developer. In this case, I believe, getting one of the low frequency errors correctly is not that important. What is important is sending the most tickets to the correct teams. In this case, accuracy is the correct answer.

More generally, your metric of evaluation has no relation with what tricks you use in your classifier. If the problem requires that you use F1, use it, regardless on which tricks/techniques you used internally to deal with the imbalanced problem.

$\endgroup$
5
$\begingroup$

If you want to include accuracy as a performance metric, balanced accuracy is a better choice than accuracy because of the imbalance in class distributions here. I would also recommend reading these answers about why accuracy is generally a bad metric.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.