I have trained a Random Forest model in python. The results are decent, and I am fairly happy with it. The input data is fairly big: ~3,000,000 observations and ~2,000 features. For various reasons, I do not want to reduce the number of features.
In producing results, I extract the
feature_importance_ from the results of the model.
I am wondering, are these importance values in themselves additive? I know the value is not additive for the feature's underlying value in a linear way (twice as much of Feature X does not make it twice as important). But could I take, say, two features, add the importance values, and say this combination of features is more important than any single item in of those three.
For example, say I have selected these three features for some reason:
Could I say that
Feature 24 + Feature 75 == .04 + .03 = .07 is more important than
Feature 10 == 0.06 alone simply because of the addition of importance?
For a little more context, I have built a model that has features that a client requested all be included and all be separate. The question then becomes not only what the important features are on their own, but what combinations might be important. Note that I don't need an absolute importance value for its own sake, but a way to rank sets of the results.