1
$\begingroup$

I created a random forest model with 17 predictor variables and a categorical outcome with 3 classes (650 data points). When I look at the variable importance plot, I see that the ranking of variables is quite different depending on whether accuracy or gini index is used as the ranking metric. What might be the reason behind ranking discrepancies between these two methods?

enter image description here

$\endgroup$
1
$\begingroup$

there can be many different combinations of reasons at play. I'll try to list them down and perhaps you can check which one is valid for your problem as the details for the data(features, target) etc. is not known

  • Mean Decrease in Impurity(MDI) can be biased towards categorical features which contain many categories
  • Mean Decrease in Accuracy(MDA) can provide low importance to other correlated features if one of them is given high importance

As you can see the definition of both the metrics is very different and both are susceptible to some kind of errors as per the data distribution. You can read more about them here and here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.