3
$\begingroup$

Ranger documentation states that if the importance mode is set to 'Impurity', then the estimated measure is '...the variance of the responses for regression..." Could someone expand on this or provide a relevant publication? As a starting point for an answer, I'm assuming it is something like the the sum of all the differences in response variance between nodes pre/post split where the feature of interest is used...maybe normalized by the number of trees?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, it's the weighted variance of the response in the two child nodes. We don't need the pre-split variance for optimization because that won't change with potential split points.

With notation of Ishwaran 2015 Eq. (2):

$$\hat{D}(s,t) = \hat{p}(t_L) \hat{\Delta}(t_L) + \hat{p}(t_R) \hat{\Delta}(t_R)$$,

where $t$ is the current node, $s$ is the split point, $\hat{p}(t_L)$, $\hat{p}(t_R)$ are the proportion of data in the left/right child node and $\hat{\Delta}(t_L)$,$\hat{\Delta}(t_R)$ are the response variances in the child nodes.

You can also call it sum of squares splitting, since the denominator cancels out if you include the current node variance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.