Ranger documentation states that if the importance mode is set to 'Impurity', then the estimated measure is '...the variance of the responses for regression..." Could someone expand on this or provide a relevant publication? As a starting point for an answer, I'm assuming it is something like the the sum of all the differences in response variance between nodes pre/post split where the feature of interest is used...maybe normalized by the number of trees?
Clarification on variable importance (i.e., impurity mode) for Ranger random forest regression model
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, it's the weighted variance of the response in the two child nodes. We don't need the pre-split variance for optimization because that won't change with potential split points.
With notation of Ishwaran 2015 Eq. (2):
$$\hat{D}(s,t) = \hat{p}(t_L) \hat{\Delta}(t_L) + \hat{p}(t_R) \hat{\Delta}(t_R)$$,
where $t$ is the current node, $s$ is the split point, $\hat{p}(t_L)$, $\hat{p}(t_R)$ are the proportion of data in the left/right child node and $\hat{\Delta}(t_L)$,$\hat{\Delta}(t_R)$ are the response variances in the child nodes.
You can also call it sum of squares splitting, since the denominator cancels out if you include the current node variance.
