Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc).
I train an XGBoost model on a numeric target Y after having performed one-hot encoding on the M categorical variables, thus creating a set of dummy inputs.

When looking at the model results, I get a table of importance gain for the categories of each feature, meaning how important they are in the model. A toy result would look like this:

feature | category               gain
       var1 | cat3                 25
       var2 | cat1                 20
       var1 | cat5                 12
       var5 | cat6                 11
       var4 | cat1                  8
           ...                    ...

The main question I'm asking is the following:

  • In order to get an idea of how important a variable is overall rather than just one of its categories (for instance, how much var1 is important overall rather than just category cat3 of var1), does it make sense to take the average of all the importance gains for each feature as an importance indicator?

Probably the sum of such gains would not be correct as the features may have a different number of categories, but I'm wondering if the average of such gains might serve as an indicator of the importance of a particular feature overall.

I already looked at some questions like this and this without gaining much decisive insight about this topic.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.