0
$\begingroup$

I am doing some tests with randomForest and I cannot understand the following results :

set.seed(12345)
library(caret);library(randomForest)
rmse_vec1<-rmse_vec2<-cor_vec1<-cor_vec2<-c()
for (i in 1:50)
{
  df_train<-data.frame(x1=rnorm(1000),x2=rnorm(1000),x3=rnorm(1000),x4=rnorm(1000))
  df_train$y<-jitter(df_train$x1+df_train$x3,amount=2)

  df_test<-data.frame(x1=rnorm(1000),x2=rnorm(1000),x3=rnorm(1000),x4=rnorm(1000))
  df_test$y<-jitter(df_test$x1+df_test$x3,amount=2)

  rf1<-randomForest(y~x1+x3,df_train) #First model
  p1<-predict(rf1,newdata=df_test)

  rf2<-randomForest(y~x1+x2+x3+x4,df_train) #Second model
  p2<-predict(rf2,newdata=df_test)

  cor_vec1<-c(cor_vec1,cor(p1,df_test$y))
  cor_vec2<-c(cor_vec2,cor(p2,df_test$y))
  rmse_vec1<-c(rmse_vec1,RMSE(p1,df_test$y))
  rmse_vec2<-c(rmse_vec2,RMSE(p2,df_test$y))
}

length(which(cor_vec2>cor_vec1)) # 47
length(which(rmse_vec2<rmse_vec1)) # 42

In 94 % (for correlation) and 84 % (for RMSE) of tests, y ~ x1 + x2 + x3 + x4 is doing better than y ~ x1 + x3 even if y <- x1 + x3 + noise !

Could anyone explain me this result ?

$\endgroup$
0
$\begingroup$

probably because you're diluting your data with normally distributed (same mean and variance I assume) data which has 100% correlation to itself, which it turn improves your total correlation.

$\endgroup$
  • $\begingroup$ Thank for your answer. Could you elaborate a little on this ? Because it is still unclear to me how it impacts my results. $\endgroup$ – MassCorr Sep 4 '18 at 14:04
  • $\begingroup$ I just replaced every rnorm with runif in my code and it does not change anything. $\endgroup$ – MassCorr Sep 4 '18 at 14:07
  • $\begingroup$ replace it with rnorm with different mean and variance for each variable, also maybe try increasing the amount of random variables $\endgroup$ – Gomunkul Sep 4 '18 at 14:08
  • $\begingroup$ Indeed with different mean and var for each predictor y~x1+x3 is doing better than y~x1+x2+x3+x4. But I still dont understand clearly these results. $\endgroup$ – MassCorr Sep 4 '18 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.