I am working on a ML problem incorporating four data streams, each producing multiple features.
We would like to know if each of the data streams provides a significant addition to the model performance.
If a data stream contribution is low, and upkeep cost is high, we should cut it from our model.
I have been thinking about what feature-importance technique to use, but most techniques are focussed on single features, like "Leave-One-Feature-Out" importance calulation 1.
However, in this problem we only have 4 data streams, and thus only 15 possible configurations. This is why I am considering what is basically an exhaustive search: Training a model for each of 15 combinations and sort results by data stream.
It sounds a but brute to me, but is there a term/name for this type of approach? I would love to read more on this type of testing.