1
$\begingroup$

I am using random forests to generate regression trees. My dataset has 30k observations of 15 variables.

Each tree I generate is limited to a 4 nodes depth (including terminal leaves) and to a sampling of 6 variables amongst the 16.

I get that kind of tree: enter image description here

Problem here is the 2 variables DPS_g_0 and DPS_g_1 are used several time in the same branch. It means that the algo can't find a better variable to split the subset, than the same it has used on the upper level.

Is using that kind of tree a sign of an overfitted random forest?

Thanks for your help

$\endgroup$
1
$\begingroup$

(1) no - it doesn't mean you're overfitting
(2) would use a test set if you're concerned about overfitting rather than looking at individual trees from the forest.
(3) trees need to use variables repeatedly to model any function other than the simplest step function
(4) usually one doesn't limit tree depth when using random forest
(5) looks like you using pary package and one assumes cforest. Thought this doesn't make a big difference for this issue, it is slightly different from randomForest implementation.

$\endgroup$
  • $\begingroup$ Hi charles. I indeed used party I've tried ranger package lately which gives OOB stats with quite good results. Thanks for your answer $\endgroup$ – Ben Dec 23 '15 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.