A decision tree can result in leaf nodes that have samples from multiple classes. Is the algorithm at that point to simply vote on the class?


Yes, one option is to simply output the majority class in each leaf node. Another option is to output a probability distribution over classes for each leaf node (this approach is sometimes called a 'probability estimation tree'). Relative class frequencies are the simplest way to estimate this distribution, but smoothing approaches have also been described in the literature (e.g. add 1 to the class counts, then divide by the number of points plus 1). Probability calibration is also possible (e.g. using Platt scaling or isotonic regression).

Some advantages of probabilitic classifiers are: 1) They provide information about uncertainty. 2) They give better separation between the statistical and decision theoretic aspects of the problem. After learning a model, class probabilities can be transformed into hard predictions using decision rules that take various consequences into account (e.g. maybe false positives and false negatives matter differently). 3) They can be useful in various specialized contexts (e.g. ranking).

  • $\begingroup$ Great, thanks for following me to my second question :P $\endgroup$
    – allstar
    Oct 3 '18 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.