1
$\begingroup$

Instead of modeling the function as an ARIMA process, I am trying to use random forests and gradient boosting as regression techniques. In the problem setup, the predictors are t_2, and t_1 and the predicted variable is t. Using the period between January 1974 and December 1978 as a training set, and using 1979 as a test set. I am trying to make a Random Forest model in R, but I am stuck at the below part, where I am getting No. of variables tried at each split: 1 and sometimes OOB is 100%. Could anyone please help me out? Thanks In advance

library(randomForest)
library(caret)
library(e1071)
library(ranger)
#converting ldeath timeseries to vector
ldeathsVector <- as.vector(ldeaths)
#dividing the vectors into 3 vectors 
t_2<- ldeathsVector[1:70]
t_1<- ldeathsVector[2:71]
t<-ldeathsVector[3:72]
ldeathDataFrame <- data.frame(t_2=t_2,t_1 =t_1,t=t)
ldeathTraining <- ldeathDataFrame[1:58,]
ldeathTesting <- ldeathDataFrame[59:70,]
ldeathTraining <- ldeathTraining[complete.cases(ldeathTraining),]
ldeathTesting <- ldeathTraining[complete.cases(ldeathTesting),]
formula <- paste(t, "~", paste(t_1, t_2 = " + "))
ldeathRandomForest <- randomForest(t ~ t_1+t_2, 
                     ldeathTraining, 
                     num.trees = 510,importance = TRUE)
$\endgroup$
3
$\begingroup$

Random forests generate trees using a random subset of predictor variables at each split, and more typically they are used where there is a large number of predictors. Part of the robustness of random forests (for datasets with many more predictors) comes from this random selection, repeated many times.

The function's default for the number of variables to use in each tree is the square root of the number of variables for classification, or the number of variables divided by three for regression. As you only have two predictors, this works out as 1. I suggest you try passing in this parameter (mtry) as 2 so that both are included each time:

ldeathRandomForest <- randomForest(t ~ t_1+t_2, 
                 ldeathTraining, 
                 num.trees = 510,importance = TRUE, mtry=2)
$\endgroup$
  • $\begingroup$ I have tried this but it is giving me a huge Mean Square Residuals : 143703.1, and I think that is wrong, is there anything which I can do? $\endgroup$ – PSK0007 Dec 6 '18 at 23:57
  • $\begingroup$ @PSK0007 It's difficult to judge whether this is high without knowing your dataset (e.g. the order of magnitude of the response variable, whether there are outliers in the reponse variable whose influence might be reduced by transformation, whether the predictors actually do have any predictive value at all). Assuming that it is genuinely larger than realistically expected, have you ensured that you've run enough trees to reach convergence? This might provide some guidance for that. $\endgroup$ – Meghann Mears Dec 7 '18 at 13:13
  • 1
    $\begingroup$ @MeghannMears Small correction, random forest uses a different random subset of variables for every split in a tree, not one subset of the for the entire tree. $\endgroup$ – bi_scholar Jan 2 at 0:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.