Why Does a Monotonic Transformation Of Dependent Variable Change Variance Explained In Random Forest

Question

I am working with the Boston data set in R.

I have read that random forest should be able to deal with untransformed data. In my example I do a log transformation of the dependent variable. My variance explained goes from 54% to 94%.

Here is what the original dependent variable looks like.

I run a random forest model without any transformation to the data set. Our dependent variable is crim.

library(randomForest)
randomForest(crim ~., Boston)
#variance explained is 54%

Once I do a log transformation of crim(to make the distro more normal) the variance explained jumps to 95%!!

randomForest(log(crim) ~., Boston)
#variance explained is 95%

Since this is a monotonic transfromation. I did not think it would have an impact on a tree based model. Can someone provide some intuition on why this might be the case? Thank you

Answer 1

It doesn't matter that the random forest model happens to be built from a collection of binary tree splits. In your examples, the first random forest model makes predictions in the original scale and the second in a log-transformed scale. Whether the predicted values are obtained from a standard linear regression or a random forest regression, the issue is how close the predictions come to the actual values in the scale of the transformation used in the model.

As this answer says, the "percent variance explained" is 100 times the pseudo-$R^2$ from the random forest regression model. As this answer shows, that pseudo-$R^2$ is given by: $$ R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2} . $$

where $y_i$ are the observations, $\hat y_i$ are the predicted values, and $\bar y$ is the mean of the observations.

So if a transformation brings the predicted values $\hat y_i$ relatively closer to the observations $y_i$ in the transformed scale over what was seen in the original scale, the $R^2$ and the "percent variance explained" will be higher in the transformed scale.