2
$\begingroup$

I've built a random forest model (regression model) using randomForest package in R, and I calculate the correlation between the predicted values and the actual ones in order to know how the trained model is going to perform, which is very high in my case, so I was wondering in such case how this correlation is build, I mean is it leave one out correlation or any type of cross validation correlation or just random and can't represent the real performance of the model when tested on unseen new cases???? the following is a snapshot of my script for calculating the correlation where x is the data (observations) and y is the numeric values I want the model to learn/predict (in the testing cases):

mytr_all = randomForest(x, y, ntree = 500,corr.bias=TRUE)
cor(mytr_all$y,mytr_all$predicted)
$\endgroup$
  • $\begingroup$ You should note that mytr_all$predicted are the out of bag (OOB) estimations of your random forest model so a high correlation between that and the observed values is a good hint that the model is not working poorly. $\endgroup$ – JEquihua Apr 19 '14 at 16:39
1
$\begingroup$

According with the R manual the cor function, with default parameter values computes Pearson correlation. A detailed description of Pearson correlation you can find on the dedidated Wikipedia page.

What is important to note here is that Pearson's correlation coefficient is a measure of linear correlation. So a value close to $1$ or $-1$ shows only that there is a linear dependence between the actual values of $y$ and your prediction $\hat{y}$.

It is a known fact that RF has some bias on regression. That mostly happens due to averaging on leaves, but I think it is a longer discussion.

However the value of the Pearson correlation coefficient might give you some hints on how the prediction bias looks like (you need also a graphical inspection of your residuals, at least), but it is certainly not a measure for accuracy.

$\endgroup$
1
$\begingroup$

If you refer to the 'cor' command, it computes simply the Pearson correlation coefficient between all predicted and observed values in the respective set. Other types of correlation are also available.

If we assume the data are generated by a classical "signal+noise" model of the form $$ Y=f(X)+\epsilon $$ where $f$ is the "true", unknown function and $\epsilon$ is a random error term, the random forest algorithm finds a function $\hat{f}$ which approximates $f$ (just like any nonparametric regression method). The predicted value for any argument $x$ is then $\hat{y}=\hat{f}(x)$. If the model is close to the unknown "truth", then the relationhip betwenn $\hat{y}$ and observed $y$ should be approximately linear, and therefore the Pearson correlation is a very good measure of its strength.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.