# Improve performance for weak class in multi-class classification

In a multi-class classification problem, models that I've trained (and which I intend to use in a majority voting ensemble) consistently get weak performance on a couple of specific classes. There is class imbalance, but the "bad" classes are not the least frequent classes. What strategies are there to boost performance specifically for those classes?

I had an idea but I'm not quite sure how to implement it: train an additional model that does binary classification on the weak class vs all others, then add that to the majority vote (either raw votes or probabilities) ensemble to give stronger weight to that class when other models get it wrong. Is this a reasonable approach?

• Among their other mechanics, boosting methods like Adaboost increase the weight of misclassified points and decrease the weight of correctly classified points on each iteration. So, future classifiers in the ensemble concentrate more on the points misclassified so far. Although this works at the level of individual points rather than entire classes, it might do what you want if points from one class are systematically misclassified. But, it can increase sensitivity to outliers. You could try boosting your classifier, or think about how to borrow/modify this particular feature. – user20160 Jan 25 '17 at 20:53
• @user20160 Thanks, I'll try an Adaboost method too. I've already tried xgboost and it was one of the better performers, but it still did poorly in the "bad" classes. – adatum Jan 25 '17 at 21:17

• I haven't tried modified loss functions yet; any tip/resource on how to do that in R? I'm using caret and I did try over- and under-sampling, and SMOTE, but they all resulted in worse performance. Also, does my additional model idea seem valid? – adatum Jan 25 '17 at 19:11