Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

Question

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more appropriate measure, but I sometimes encounter cases where the RMSE is acceptable but the R-Squared is way low ~ 0.18 for example in a recent test. How should one interpret this ?

Also, re: tuning parameters in GBM - I use caret in R and as usual I set a range of values for tree (e.g. 1:100), for interaction depth and so on. I need to check the relative influence of the variables once the GBM is run and in different levels of tuning (eg., shrinkage = 0.01 vs shrinkage 0.001) the rel. influence of the variables can become "very" different. Is this indicative of something fundamentally concerning the data or conversely, how does everyone generally address such an issue (i.e, at what point would you stop tuning further given each run takes a long time (large datasets of x 100k rows) and consider a model as with a given set of tuning parameters as final). Although GBM selects the optimal n trees and depth, finer tuning can change that selection, and consequently alter the results of the variable importance. Same question also for Random Forests.

Thanks in advance for your thoughts on this.

Answer 1

If you calculate $R^2$ as the squared Pearson correlation between observed ($y$) and predicted outcomes ($\hat y$), $R^2=\left( \text{corr}\left( y, \hat y \right) \right)^2$, there are some major issues. In particular, $\left(\text{corr}\left( y, a+b\hat y \right) \right)^2=\left(\text{corr}\left( y, \hat y \right) \right)^2$ for $a\in\mathbb R$ and $b\ne 0$, meaning that you can shift the predictions up and down with $a$ or scale them with $b$, and this calculation will not catch that. As an example, $y=(1,2,3)$ and $\hat y=(-105,-205,-305)$ have a perfect squared correlation of $1$ between them, yet the predictions $\hat y$ of $y$ are clearly terrible. Consequently, this probably is not the calculation you want to use.

An alternative calculation, equal to the squared Pearson correlation in OLS linear regression, is below.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The numerator is equal to $n\times(RMSE)^2$, so this is just a monotonic (decreasing) function of the RMSE…

…if the denominator is constant, which is unlikely when you are doing cross validation. Therefore, this $R^2$ involves more than just the performance of your model, introducing a complexity that is difficult to resolve (especially when you consider that it is not clear what calculation to perform when it comes to holdout data). Thus, if you are getting acceptable $RMSE$ values, especially on holdout data, that should be a signal that your model is performing the way you want it to, regardless of what is happening with the ambiguous $R^2$ value you calculate.