I have a heavily imbalanced multiclass text classification problem: one class is very probable a priori (
P), while the remaining four ones are about equally improbable (
I4). I have performed many experiments using a single multiclass Random Forest, trying to model all classes at once (with the goal of optimizing the negative loglikelihood).
The heavily imbalanced structure of that particular problem made me think however that a better strategy would be to decompose it in a hiercharchy, by conflating the improbable classes into one (
I): a first-level RF could be used to model the
I binary problem, while a second-level RF could focus on the specialized four-class problem. It's then easy to combine the outputs of the two models into one.
My intuition was that this decomposition should work well, because each model has a simpler task than the overall one, with more training information (i.e. being less diluted in a wider and unevener set of classes). But even though I've been very careful to make my two implementations very similar and comparable, I found that the single multiclass model clearly outperforms the hierachical one, contrary to my intuition. I'd like to know if there's a way to understand this result.