Is feature importance from Random Forest models additive?

Question

I have trained a Random Forest model in python. The results are decent, and I am fairly happy with it. The input data is fairly big: ~3,000,000 observations and ~2,000 features. For various reasons, I do not want to reduce the number of features.

In producing results, I extract the feature_importance_ from the results of the model.

I am wondering, are these importance values in themselves additive? I know the value is not additive for the feature's underlying value in a linear way (twice as much of Feature X does not make it twice as important). But could I take, say, two features, add the importance values, and say this combination of features is more important than any single item in of those three.

For example, say I have selected these three features for some reason:

Feature: Importance: 10 .06 24 .04 75 .03

Could I say that Feature 24 + Feature 75 == .04 + .03 = .07 is more important than Feature 10 == 0.06 alone simply because of the addition of importance?

For a little more context, I have built a model that has features that a client requested all be included and all be separate. The question then becomes not only what the important features are on their own, but what combinations might be important. Note that I don't need an absolute importance value for its own sake, but a way to rank sets of the results.

Answer 1

No. The importance is measured by mean decrease Gini and it is not additive.

We can easily work out an example to show it is not additive. Assume we have two identical features, both of them are important / strongly correlated to the label.

It is easy to see they are equally important but not additive (which means using both features will get "double benefits").

> library(randomForest)
> d=mtcars[,c('mpg','wt')]
> d$wt2=d$wt
> fit=randomForest(mpg~wt+wt2,d)
> importance(fit)
    IncNodePurity
wt       508.7214
wt2      542.0116