6
$\begingroup$

Is it necessary to use cross-validatation to avoid overfitting when applying random forest algorithm?

This is a question asked in my recent data scientist interview. Can anyone give any ideas? Thanks

$\endgroup$
0

2 Answers 2

4
$\begingroup$

Well, random forest uses bagging which is specifically designed to reduce problems with overfitting.

Ensemble methods like bagging and CV are both ways to avoid overfitting.

Cross-validation can be used in random forest modelling various ways - e.g. to find the optimal number of trees - but I don't know anywhere it has to be used. For example, to measure the out-of-sample performance I think you can use the out-of-bag error.

I suppose the resulting question is 'can overfitting - while reduced in scope - still be a problem if you don't use cross-validation'? I'm not 100% certain of the answer to that, but searching around$^{[1]}$ it looks like overfitting might still be a potential issue (BMA and bagging are both forms of model averaging, the problem could easily carry over to bagging and so perhaps to random forests). In that case, some other approach - such as cross-validation - might be needed.

(Cross validation isn't the only other way to reduce/avoid overfitting of course, which may have been the underlying point of the question.)


[1] Domingos, P., (2000)
"Bayesian Averaging of Classifiers and the Overfitting Problem"
Proceedings of the Seventeenth International Conference on Machine Learning, pp.223-230

$\endgroup$
1
$\begingroup$

As random forests is working on the concept of Bootstrap aggregating, there is no special need for cross validation. while dealing with large number of trees in forest, cross validation will take much of your time.

And Glen_b also mentioned that, CV and Bagging are two approaches to reduce overfitting, so using one of them will be just fine.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.