2
$\begingroup$

I've noted that quite a few of the node splits in my Random Trees are redundant since both child nodes have identical classifications. I suspect this is caused by an unfortunate choice of parameters, but which?

I'm using the OpenCV implementation to generate the trees; for prediction I use my own code. That's how I discovered these redundant nodes. The OpenCV trainings parameters are:

  • max_depth = 20
  • min_sample_count = 7
  • nactive_vars = 2
  • max_num_trees = 250
  • forest_accuracy = 0
  • term_crit = CV_TERMCRIT_ITER (i.e. 250 trees)

I have +/- 50.000 samples, 25 variables per sample and 7 classes. No missing variables, no categorical variables. The problem with redundant nodes appears with nodes much shallow than max_depth so that's probably not relevant.

$\endgroup$

migrated from dsp.stackexchange.com Jan 24 '14 at 3:11

This question came from our site for practitioners of the art and science of signal, image and video processing.

2
$\begingroup$

This is a normal RF behaviour... The idea is that splits are made greedily, i.e. if a node does not contain only one class it will be always split in hope that this will lead to some meaningful separation downstream thanks to the simplification of the problem.
Sure, such redundant splits can be easily removed without altering RF results by re-creating each tree, but this is not a part of a canonical RF algorithm as it was designed for accuracy and simplicity rather than computational efficiency.

$\endgroup$
1
$\begingroup$

If you look in the XML file (produced from training), you can figure out what the variable is that you are interested in that gives you poor splitting ability. Additionally, you can perform a post-analysis to identify the variable importance which may allow you to eliminate those particular variables that give poor splitting ability.

$\endgroup$
  • $\begingroup$ I have a .yaml file which is >3GB. Aside from being impractical, how would I identify such a variable? Is it even consistently causing problems? I already choose my variables based on their importance (as indicated by OpenCV learning algorithm) $\endgroup$ – MSalters Feb 6 '14 at 9:11
  • $\begingroup$ There could be a variety of reasons that you get redundant splits. Most of the time, it's related to the set of termination criteria (i.e. min_samples, max_depth, etc). When you find a redundant split, can you see how many samples there are in the left/right splits and the depth in which they split? $\endgroup$ – slaw Feb 7 '14 at 4:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.