Till now I have used a following flow for training a random forest model.

create 10 folds of data.
for each fold i:
    - use ith fold as validation data
    - use remaining 9 folds as training data
    - apply normalization on training and validation data
    - # apply feature selection on training data
    - # select same features from validation data
    - train random forest on training data
    - predict values for validation data
combine all predictions.

Now I want to do feature selection using varImp() function. I am confused as it is said that varImp itself trains a model on training data to find out best set of features.

How should I use varImp to get important features (say using partial least squares) and then again apply training model on training data?

  • $\begingroup$ Inside your loop, step/line 4 says "apply feature selection", how did you select these features? $\endgroup$
    – horaceT
    Sep 12, 2017 at 1:13
  • 1
    $\begingroup$ There's no need to use feature scaling with Random Forest. stats.stackexchange.com/questions/255765/… If you're using random forest, you might as well apply a random forest-based feature selection method such as the Boruta algorithm. stats.stackexchange.com/questions/2350/… $\endgroup$
    – Sycorax
    Aug 13, 2018 at 0:11

2 Answers 2


This sounds a lot like recursive feature elimination. See the caret help page for feature selection.


I'd suggest removing the least important/lowest varImp features using this criterion, then re-runing test and validation sets. In some cases, but not always, this would increase your success. Note that normalization is generally not recommended for this approach and may in fact result in a loss of valuable signals.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.