An overview of popular tree based models

Question

There are many different widely used tree models. This is my notes and understanding of them. Could anyone tell me

• If I am correct on the model I mentioned
• If I am missing other "popular" tree model

---

Statistic community (many packages can be found in R)

• CART (Classification And Regression Trees). This is from the classical book Classification and Regression Trees by Breiman. This is trademakred name of particular software implementations of the ideas.
• Tree package in R. Tree has been used for S Plus routines of Clark and Pregibon.
• RPART package in R. Free version of CART, and widely used in R, because of free it is even more popular than CART
• PARTY. Conditional Inference Tree Model

Data mining community (many packages can be found in Java)

• ID3 (Iterative Dichotomiser 3), Invented by Ross Quinlan. ID3 is the precursor to the C4.5 algorithm, and is typically used in the machine learning and natural language processing domains.
• C45, is an extension of ID3 that accounts for unavailable values, continuous attribute value ranges, pruning of decision trees, rule derivation, and so on.
• J48, is a re-implementation of C4.5 release 8 (hence the name J48) in Java. A lot of time has been spent getting the same results as the original C4.5. J48 implements both C4.5's confidence-based post-pruning (default) and sub-tree raising.

Answer 1

CART and ID3 are classical decision tree algorithms. You can also investigate using which measure they are doing splits - there can be Gini index or cross-entropy measuring information gain.

But for practical tasks I think pure trees are rarely used as there are Random Forest or (Gradient) Tree Boosting algorithms showing greater performance, latter uses "dumb" decision stumps which are short trees with a very small number of splits (only 1 or 2 for example).