In a multi-class classification problem, models that I've trained (and which I intend to use in a majority voting ensemble) consistently get weak performance on a couple of specific classes. There is class imbalance, but the "bad" classes are not the least frequent classes. What strategies are there to boost performance specifically for those classes?
I had an idea but I'm not quite sure how to implement it: train an additional model that does binary classification on the weak class vs all others, then add that to the majority vote (either raw votes or probabilities) ensemble to give stronger weight to that class when other models get it wrong. Is this a reasonable approach?