# random forest regression predicts "opposite"

I have a dataset with 70 features, which are continuous measures and are interrelated but not highly correlated ($$|\rho| <.5$$. I have several outcomes, which are each integer values ranging from 0-80. For each outcome I perform the forllowing steps:

1. I perform a random 50/50 train/test split.
2. I fit a RF model using the randomForest package in R using the default settings.
3. I predict score using the test data and calculate Pearson correlation between the predicted values and the outcome in the test set.
4. I repeated this 100 times.

What I get get is a distribution of Pearson $$\rho$$ values that indicate the accuracy of the model where highest correlation of 1 means that the model perfectly predicts and 0 means the models does not predict at all.

What I get for some outcomes is a distribution of Pearson $$\rho$$ values where the distribution is smaller than 0. Thus, the model predicts basically "the opposite". I cannot make sense of this and therefore wanted to ask here what that means or how this could be? How can the correlation be significantly negative? If the model does not perform well or there is nothing in the data it should be randomly distributed or distributed around 0 but not significantly negative.

EDIT: I tested tuning mtry. Even though this improves out of sample accuracy for all models that worked anyways, it does not change the problem that some models predict the opposite. I also tried extremely random forests but get the same problem.

• There's nothing magical about the default settings. Indeed, it could be that the trees are too rich and the ensemble is overfitting. What happens when you tune the model?
– Sycorax
Apr 28 '19 at 20:01
• Did you randomly split the data into test and training sets? Apr 28 '19 at 21:13
• @Ozan yes, I did random split using the sample function in R. Apr 29 '19 at 7:14
• @Sycorax I did not try to tune the model because in my field of research the standard model is used and it can be difficult to get through if you deviate from that. I will check it out but I need some time for that. Apr 29 '19 at 7:14
• What exactly can you not make sense of? Sounds like your models just don't work very well. May 24 '19 at 12:21

You write:

I repeated this 100 times.

• I see the idea behind your answer but I am not sure if I get it in this particular context. Lets say I generate a random variable x and y, calculate Pearson r. Then 5% of the times should be significant. But in my case: I compute a correlation between actual and predicted values. Thus, between $y_i$ and $\hat{y}_i$ whereby the model to predict used a random subset of the data. I did this btw now 1000x instead of 100 with similar results. n is 100-150 depending on the outcome. Would you still say that the entire cloud of 1000 correlations could be negative? May 30 '19 at 11:52