random forest regression predicts "opposite"

I have a dataset with 70 features, which are continuous measures and are interrelated but not highly correlated ($$|\rho| <.5$$. I have several outcomes, which are each integer values ranging from 0-80. For each outcome I perform the forllowing steps:

1. I perform a random 50/50 train/test split.
2. I fit a RF model using the randomForest package in R using the default settings.
3. I predict score using the test data and calculate Pearson correlation between the predicted values and the outcome in the test set.
4. I repeated this 100 times.

What I get get is a distribution of Pearson $$\rho$$ values that indicate the accuracy of the model where highest correlation of 1 means that the model perfectly predicts and 0 means the models does not predict at all.

What I get for some outcomes is a distribution of Pearson $$\rho$$ values where the distribution is smaller than 0. Thus, the model predicts basically "the opposite". I cannot make sense of this and therefore wanted to ask here what that means or how this could be? How can the correlation be significantly negative? If the model does not perform well or there is nothing in the data it should be randomly distributed or distributed around 0 but not significantly negative.

EDIT: I tested tuning mtry. Even though this improves out of sample accuracy for all models that worked anyways, it does not change the problem that some models predict the opposite. I also tried extremely random forests but get the same problem.

• There's nothing magical about the default settings. Indeed, it could be that the trees are too rich and the ensemble is overfitting. What happens when you tune the model?
– Sycorax
Commented Apr 28, 2019 at 20:01
• Did you randomly split the data into test and training sets? Commented Apr 28, 2019 at 21:13
• @Ozan yes, I did random split using the sample function in R. Commented Apr 29, 2019 at 7:14
• @Sycorax I did not try to tune the model because in my field of research the standard model is used and it can be difficult to get through if you deviate from that. I will check it out but I need some time for that. Commented Apr 29, 2019 at 7:14
• What exactly can you not make sense of? Sounds like your models just don't work very well. Commented May 24, 2019 at 12:21

You write:

I repeated this 100 times.

• I see the idea behind your answer but I am not sure if I get it in this particular context. Lets say I generate a random variable x and y, calculate Pearson r. Then 5% of the times should be significant. But in my case: I compute a correlation between actual and predicted values. Thus, between $y_i$ and $\hat{y}_i$ whereby the model to predict used a random subset of the data. I did this btw now 1000x instead of 100 with similar results. n is 100-150 depending on the outcome. Would you still say that the entire cloud of 1000 correlations could be negative? Commented May 30, 2019 at 11:52