Feature subset selection by stepwise regression for a random forest model? I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, some of the features may be strongly correlated.
By trial and error, I have found a subset of features which seem to work well (in terms of mean squared error on the test set), but I would like to see if I can do better. I was thinking of some sort of autmatic feature subset selection, such as stepwise regression. However, I have two questions


*

*Since random forest models already select features, is it possible to gain much by such a method?

*What would be a suitable information criterion for a random forest model? Since it's nonparametric, standard information criteria (such as AIC or BIC) one would normally use for stepwise regression, don't work.

 A: 
Since random forest models already select features, is it possible to
gain much by such a method?

Yes and no. By selecting only a subset of features, and creating synthetic variables, you can help/accelerate the convergence of trees. But not necessarily improve it, because synthetic variables, which are a combination of one or more variables and one or more division rules, are nothing more than nodes in a decision tree, and so the tree will find them on its own if it finds them relevant.
But the idea is good, a widely used technique is to look at the results of a simple decision tree, and use the conditions of these first nodes to create synthetic features that we reinject in a logistic regression or stepwise regression.

(source: scikit-learn.org)

What would be a suitable information criterion for a random forest model? Since it's ninparametric, standard information criteria (such as AIC or BIC) one would normally use for stepwise regression, don't work.

The objective of each split is to find the division, or more precisely the variable and the division rule, which will contribute to the strongest decrease of the heterogeneity of the son nodes on the left $\kappa_l$ and on the right $\kappa_r$.
In the case where Y is a qualitative variable, several heterogeneity functions can be defined for a node: a criterion defined from the notion of Entropy or from the Gini concentration (actually there is also the CHAID criterion which is based on the statistical test of $\chi^2$).
In practice, it turns out that the choice of the criterion has little influence, and it is often Gini that is chosen by default.
In the node $\kappa$, where $p_\kappa^l$ is the number of element of the class $l$ in the node $\kappa$, and $m$ the number of classes.
\begin{align*}
\textit{Entropy} : 
     &S_\kappa = - \sum_{l=1}^{m} p_\kappa^l log(p_\kappa^l) \\
\textit{Gini} : 
    &G_\kappa = \sum_{l=1}^{m}p_\kappa^l(1 - p_\kappa^l) = 1 - 
                \sum_{l=1}^{m}(p_{\kappa}^{l})^2
\end{align*}
As you mentioned earlier, we cannot directly use the Akaike information criterion or the Bayesian information criterion. Nevertheless, it is possible to easily apply a backward stepwise selection.
In the case of random forests, a method for selecting variables is based on the importance score of the variables (ability of a variable to predict $Y$). We thus employ a top-down (or backward) strategy where we remove step by step the least important variables as defined in the importance criterion.
At each stage of the algorithm, we calculate the prediction error. The subset finally chosen is the one that minimizes the prediction error.
The agorithm can be summed up as follows :
1. Construct a random forest and calculate the error
2. Calculate the measure of importance of each variables
3. Eliminate the least important variable
4. Repeat steps 1 to 3 until all variables have been eliminated

A: I find Thomas's answer very detailed, but I will nevertheless add a couple things to it:

Since random forest models already select features, is it possible to gain much by such a method?

You could have some gains from feature selection in cases of highly correlated features and when having many unimportant features. Many high correlated features might degrade the performance of your trees in the sense that, since they carry the same information, every split to one of them will affect the "remaining" information in the other ones. If you therefore split early in the tree on one of these variables, if features that are correlated to it will be selected for the subsequent splits they might generate useless or "bad" splits. In the same way, when you have too many unimportant features, since you are subsampling the variables for the splits you might find yourself making multiple splits on useless variables (to my knowledge, this is particularly a problem in application in biology).

Therefore, you might want to remove/work on highly correlated features, and at the same time always re-tune your mtry parameter, as this will need to change based on the importance and number of your variables.

What would be a suitable information criterion for a random forest model?

Here I also agree with Thomas. RF will give you variable importances and these can be used to do some selection. However: you should only use variable importances if your model is strong enough. If your model is overfitting (can happen when having simple/few data) or too weak, your variable importances might fluctuate a lot from run to run. 
Also, be aware that feature importances are ALSO strongly affected by correlated features, so if you plan on using this, take it into account. 
Finally, default Variable Importances based on Gini/Entropy (or splits in general for the matter) are done on the train set and are strongly biased towards numerical features or categorical features that have many levels, since they allow for splitting more times within the same tree. For this reason, you might also want to check Permutation Importances, which are instead model agnostic and can be use on a performance metric of your choice. (More here)
