I created a random forest model with 17 predictor variables and a categorical outcome with 3 classes (650 data points). When I look at the variable importance plot, I see that the ranking of variables is quite different depending on whether accuracy or gini index is used as the ranking metric. What might be the reason behind ranking discrepancies between these two methods?
there can be many different combinations of reasons at play. I'll try to list them down and perhaps you can check which one is valid for your problem as the details for the data(features, target) etc. is not known
- Mean Decrease in Impurity(MDI) can be biased towards categorical features which contain many categories
- Mean Decrease in Accuracy(MDA) can provide low importance to other correlated features if one of them is given high importance