# Workflow design: Random forest regression to see which independent variables are more important

I want to build a very simple random forest for regression (not classification). I have one numeric dependent variable and 11 independent variables (3 of them are numeric and the rest are categorical). The number of observations after treating n/a values is 787.

This is the str of my data frame:

'data.frame' : 787 obs. of  12 variables:
$$a : num 3.02 3.02 3.02 3.02 4.02 4.02 4.02 4.02 4.06 4.06 ...$$ b : int  300000000 300000000 300000000 300000000 130000000 130000000 130000000 130000000 200000000 200000000 ...
$$c : Factor w/ 6 levels$$ d : Factor w/ 2 levels
$$e : Factor w/ 6 levels$$ f : Factor w/ 3 levels
$$g : Factor w/ 11 levels$$ h : Factor w/ 4 levels
$$i : Factor w/ 2 levels$$ k : num  2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 ...
$$l : num 74 74 74 74 192 192 192 192 119 119 ...$$ m : num  183 196 175 311 206 ...


What I want from the model is to rank in a way the independent variables based on their importance. I have some questions with regards to the steps that I need to follow.

Do I have to split my data into train and validation sets? That said is validation needed in the end or is a step that I can skip and thus work with my full data set?

I have run:

library(randomForest)
model1 <- randomForest(y ~ ., data = data_selected, mtry=3, ntree=500, importance = TRUE)
print(model1)


and this is my result:

Call:
randomForest(formula = mid_spread ~ ., data = TrainSet, mtry = 2,      importance = TRUE)
Type of random forest: regression
Number of trees: 500
No. of variables tried at each split: 2

Mean of squared residuals: 63426.82
% Var explained: 50.72


Don't you think that % Var explained: 50.72 is quite low? I have tried to tune the tree in order to find ideal mtry but improvement is marginal. The same when I try different different number of trees.

I have also used:

importance(model1)
varImpPlot(model1)


Is this enough? Could anyone give me a suggestion of what steps I need to follow in order to have a complete workflow?

• One well-known and principled way to use random forest to find relevant features is called Boruta. You might start there.
– Sycorax
Jan 3 '19 at 18:11
• Yes, use the training set from each round of the cross validation to estimate the ranks. A good method is Boruta as suggested by Sycorax. There’s an R package for that.
– user289381
Jul 4 '20 at 0:29