As we saw in this question, the recommended strategy of building a decision tree is postpruning.

The two methods for that are subtree replacement and subtree raising. At each node, an algorithm decides whether it should perform subtree replacement, subtree raising, or leave the subtree as is.

  • Subtree replacement selects a subtree and replaces it with a single leaf.
  • Subtree raising selects a subtree and replaces it with the child one (ie, a "sub-subtree" replaces its parent)

What are the pros and cons, when should we use each?

  • 1
    $\begingroup$ +1 Good question. I've never really used decision trees but it seems like method 1) will be much more aggressive, which may be desirable for extremely large trees, but method 2) will possibly preserve some subtleties that method 1) loses. Does it just come down to a trade-off between overall accuracy and computational efficiency? $\endgroup$ – tdc Feb 16 '12 at 17:09

My knee jerk response is:

  • without a measure of goodness the word "better" has no meaning.
  • I am trending to "c, none of the above" as my preferred approach

The approach to "build", if you mean build = grow, is to split on the best leaf until a stopping criterion is breached.

Like all Machine Learning tools, there are two goals for the tree, and they comprise a measure of goodness for the pruning process: maximize generalization, minimize error in representation. If you can agree to these two, in a general sense, then I can proceed with an answer.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.