# Сan a subset of features perform better than the base set

I have a theoretical question..

1. I have a model, let it be random forest

2. I take 100 candidate features and train the model

3. I select all the features that are important from the point of view of the model (important features from the random forest). Let the model select 40 features. I'll call it the basic feature set

My question is: Can it be that the model will work better if I choose some subset of the basic features set of in 5-10-20 features?

Is it worth looking for a subset of features in the base feature set, or is it already the best set by default?

• You would do well to place no confidence in the 40 features. Bootstrap confidence intervals of the 100 features will uncover the true difficulty of the feature selection task. These confidence intervals will be embarrassingly wide. Commented Jul 9, 2023 at 15:49
• I don't really understand how to calculate and analyze these confidence intervals. If you can provide an example code in R , that would be awesome.
– mr.T
Commented Jul 9, 2023 at 16:26
• An R example is here and see also this. Commented Jul 10, 2023 at 12:26

$$I(Y; X_1, X_2) = I(Y; X_1) + I(Y; X_2|X_1)\longrightarrow I(Y;X_1,X_2)\geq I(Y; X_1)$$