1
$\begingroup$

I'm familiar with how classification trees weight the impurity measure of a potential split by the proportion of observations that would fall into each child node, such as:

$$ loss = \frac{n_1}{N_m} G_1 + \frac{n_2}{N_m} G_2 $$

where:

  • $N_m$ is the number of observations in node $m$ (i.e. the parent node that will be split).

  • $n_1$ and $n_2$ are the number of observations in the "left" and
    "right" daughter nodes that will result from the split, respectively.

  • $G_1$ and $G_2$ are the impurity measures of the "left" and "right"
    daughter nodes, respectively (either Gini Impurity or Cross-Entropy).

I'm a little confused on whether this weighting applies to the regression setting as well. In the Elements of Statistical Learning (page 308) the splitting mechanism for a regression oriented decision tree is defined in equation (9.13), reproduced below:

$$ min \left( min \left[ \sum_{x_i \in R_{m1}}^n (y_i - c_{m1})^2 \right] + min \left[ \sum_{x_i \in R_{m2}}^n (y_i - c_{m2})^2 \right] \right) $$

where

$$ c_m = \frac{\sum_{i \in R_m}^n y_i }{\left| R_m \right|} $$

However, on page (309 - 310), the authors mention that, "For regression we used the squared-error node impurity measure defined in (9.15)," which seems to imply that the squared error metric is weighted by the proportion of observations that fall into each child node due to a binary split.

So, the overall question is: Does the weighting of the child nodes apply in both the regression and classification setting, or only the classification setting?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.