Why many decision trees are using Chi2 or Information Gain Ratio to split the node when they can directly use accuracy, lift or AUC?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Many users of decision trees are comfortable and familiar with using and interpreting chi-squared tests and hence find it easy to use them in connection with decision trees as well. Of course, there are other ways to capture associations between the response and the input variables, but usually no single one measure outperforms all (or at least most) others on all (or at least most) datasets. So many researchers tend to prefer the measure that they are comfortable with or that performs particularly well on their dataset.
Another reason for using chi-squared tests is that they provide a means to separate the splitting variable selection from the split point selection. The work of Loh and co-workers (e.g., on QUEST and GUIDE) has shown that this can avoid so-called variable selection bias. This bias means that many exhaustive search strategies prefer splitting variables with many possible splits. Hence, trees based on statistical inference is used in many statistical decision tree algorithms (e.g., CTree).
answered Jan 1, 2015 at 23:18
-
$\begingroup$ Doesn't Chi2 also suffer from bias toward multinominal attributes like information gain? And isn't it possible to deal with this bias by randomly splitting the data into training/testing set during the branch split and pick the branch split based on the best score on the testing set (something like C4.5 does during the prunning)? And isn't the advantage of Chi2, IG and IGR in comparison to AC, AUC,... just in the fact that the values are NOT normalized by the sample size? $\endgroup$ Commented Jan 4, 2015 at 13:44
-
$\begingroup$ No, the bias does not come from the sample size (which might more often lead to overfitting than bias) but from only considering the best possible split for each variable. If this doesn't consider from how many possible splits the best one was chosen then there is a higher probability of selecting a variable with many possibly splits by chance. One can either address this by only using a fixed number of possible splits for variable selection only (as in QUEST or GUIDE) or by computing the correct maximally selected p-value (as in MOB or also possible in CTree). $\endgroup$ Commented Jan 5, 2015 at 7:33