23
$\begingroup$

I am running a decision tree classification using SPSS on a data set with around 20 predictors (categorical with few categories). CHAID (Chi-squared Automatic Interaction Detection) and CRT/CART (Classification And Regression Trees) are giving me different trees. Can anyone explain the relative merits of CHAID vs CRT? What are the implications of using one method over the other?

$\endgroup$
21
$\begingroup$

I will list some properties and later give you my appraisal for what its worth:

  • CHAID uses multiway splits by default (multiway splits means that the current node is splitted into more than two nodes). This may or may not be desired (it can lead to better segments or easier interpretation). What it definitely does, though, is thin out the sample size in the nodes and thus lead to less deep trees. When used for segmentation purposes this can backfire soon as CHAID needs a large sample sizes to work well. CART does binary splits (each node is split into two daughter nodes) by default.
  • CHAID is intended to work with categorical/discretized targets (XAID was for regression but perhaps they have been merged since then). CART can definitely do regression and classification.
  • CHAID uses a pre-pruning idea. A node is only split if a significance criterion is fulfilled. This ties in with the above problem of needing large sample sizes as the Chi-Square test has only little power in small samples (which is effectively reduced even further by a Bonferroni correction for multiple testing). CART on the other hand grows a large tree and then post-prunes the tree back to a smaller version.
  • Thus CHAID tries to prevent overfitting right from the start (only split is there is significant association), whereas CART may easily overfit unless the tree is pruned back. On the other hand this allows CART to perform better than CHAID in and out-of-sample (for a given tuning parameter combination).
  • The most important difference in my opinion is that split variable and split point selection in CHAID is less strongly confounded as in CART. This is largely irrelevant when the trees are used for prediction but is an important issue when trees are used for interpretation: A tree that has those two parts of the algorithm highly confounded is said to be "biased in variable selection" (an unfortunate name). This means that split variable selection prefers variables with many possible splits (say metric predictors). CART is highly "biased" in that sense, CHAID not so much.
  • With surrogate splits CART knows how to handle missing values (surrogate splits means that with missing values (NAs) for predictor variables the algorithm uses other predictor variables that are not as "good" as the primary split variable but mimic the splits produced by the primary splitter). CHAID has no such thing afaik.

So depending on what you need it for I'd suggest to use CHAID if the sample is of some size and the aspects of interpretation are more important. Also, if multiway splits or smaller trees are desired CHAID is better. CART on the other hand is a well working prediction machine so if prediction is your aim, I'd go for CART.

$\endgroup$
  • 1
    $\begingroup$ (+1). Nice overview. Could you explain what "multiway splits" and "surrogate splits" are? Are multiway splits if the splits are not dichotomous? $\endgroup$ – COOLSerdash Jun 8 '13 at 21:28
  • $\begingroup$ Thanks, I edited the answer to accomodate your comments. $\endgroup$ – Momo Jun 8 '13 at 21:38
  • $\begingroup$ Nice. I love when I learn things reading really good SO answers. Great job :) $\endgroup$ – rrenaud Jun 8 '13 at 22:38
  • 1
    $\begingroup$ @Momo: Thanks very much for the updated answer. Regarding multiway splits, I've found the following interesting statement from Hastie et al. (2013) The Elements of statistical learning: "[...] While this [multiway splits] can sometimes be useful, it is not a good general strategy. [...] Since multiway splits can be achieved by a series of binary splits, the latter are preferred." I wonder if this is really as definite as they state (I'm not very experienced with machine learning) but on the other hand, their book is considered a reference. $\endgroup$ – COOLSerdash Jun 9 '13 at 20:39
  • $\begingroup$ Yes, a series of binary splits can be the same as multiway splits. They can also be different. I tend to agree with the statement. One other thing to note is that looking for split points with exhaustive search is algorithmically simpler and faster for binary splits of a given node. $\endgroup$ – Momo Jun 9 '13 at 20:49
8
$\begingroup$

All single-tree methods involve a staggering number of multiple comparisons that bring great instability to the result. That is why to achieve satisfactory predictive discrimination some form of tree averaging (bagging, boosting, random forests) is necessary (except that you lose the advantage of trees - interpretability). The simplicity of single trees is largely an illusion. They are simple because they are wrong in the sense that training the tree to multiple large subsets of the data will reveal great disagreement between tree structures.

I haven't looked at any recent CHAID methodology but CHAID in its original incarnation was a great exercise in overinterpretation of data.

$\endgroup$
0
$\begingroup$

Very complete answer. I used CHAID in a reaserch with more than 100.000 database. At this level, classification is very precised but I recomend try few times with different numbers of partitions and the less deep levels of the tree (SPSS software allows to determinate this parameters previously). This is because CHAID generates classifications trees with several grups (multisplit) and much worse if the database is big. Final tree coul'd be huge. Finally, don't forget to use the "internal control" of sample division of the database. See also the SPSS classification trees Manual available on google.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.