# Training and test sets in random forest regression

Why do I keep reading about specifying training and test sets with random forest regression? As probably obvious from this question I am new to this method, but what I thought one of the cool things about it was that this is not needed because the out of bag sample functions as the test set, i.e. the method provides its own test set. Is this just needed under specific circumstances, e.g. for more methodological studies or am I missing some very fundamental aspect?? Thank you for any clarification.

• Dividing the data into two sets is quite wasteful of sample size (as opposed to using rigorous bootstrap validation) and requires an initial sample size of at least n=20,000 to be stable/reliable. – Frank Harrell May 10 '16 at 15:24
• Thank you both. I know it was a horribly vague question, but these answers are all I needed. I don't have a specific paper to talk about, I just saw the topic of test/training samples come up everywhere, including in the options of R random forest packages and was afraid I was missing something. Frank, do you maybe have a reference for that sample size statement? – Johannes May 12 '16 at 1:17
• I don't have a reference but base this on two things: (1) A dataset with n=17,000 and proportion $Y=1$ of 0.3 resulted in unstable $c$-index (ROC area) when using split-sample validation twice. (2) Single 10-fold cross-validation has insufficient precision; you need to repeat 50-100 times for stability. And a single 10-fold cross-validation is much better than 2-fold cv. – Frank Harrell May 12 '16 at 11:42