2
$\begingroup$

I am struggling to understand how decision trees work. I understand that you need to calculate the Gini coefficients for the sample features and that's how leaves are chosen.

My issue is that I don't understand how to find which leaf node corresponds to which class. Furthermore, how does this algorithm deal with multi-class classification?

New contributor
user341708 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
5
$\begingroup$

After splitting with respect to some criterion (e.g. gini, information gain etc.) the samples you're left with in the leaf node decides the class. Typically, the majority class is chosen, or in some situations, a probability might be more preferable (e.g. a probability vector). It doesn't matter if the problem is multi-class or binary.

$\endgroup$
4
$\begingroup$

The decision tree makes binary splits. Each node is an if statement, that leads you to the branch on the left, or on the right. When training the decision tree, the branching continues until hitting some stopping criterion (e.g. minimum number of samples per leaf). The final leaf is where you make the classification decision. You can either calculate the probability by looking at the empirical frequencies (3/7 samples at the leaf are cats) or make a decision (since 4/7 samples are dogs, this leaf classifies as "dog"). In multiclass classification, you just use majority vote. For regression problems, the prediction is just the mean of the leaf.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.