2
$\begingroup$

i have been wondering for some time now how random forests (or AdaBoost, doesn't matter) are built when using cross-validation. Let's see we're using 5-fold cross validation to train random forests on 5 different training sets and therefore test on 5 different test sets. How does the 'final' random forest look like when we are basically building 5 random forests (one for each fold of the cross validation). How are these forests combined into a final model?

I have never understood this step and I really hope someone can help me with this!

thanks in advance, Steven

$\endgroup$
1
  • $\begingroup$ You train it on the training data to get a final model. Cross-validation is just to make decisions about the parameters/models etc. $\endgroup$
    – Michael M
    Jul 25 '20 at 10:46
0
$\begingroup$

I am not sure why you are using cross-validation with RandomForest. RandomForest does not need cross validation. When you train a RF model, each tree uses bootstrapped samples from original data as train set and leaves about 1/3 of data called out of bag(oob) data. Each oob data not used in training is marked and then used for validation using the forest(data is tested on forest not on a tree). Out of bag data is used for each tree to take vote on and finally we average them to get the final result.

Random Forest does not need cross-validation to avoid overfitting. It uses (bootstrapping + averaging) called as bagging to deal with overfitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.