0
$\begingroup$

I want to build a very simple random forest for regression (not classification). I have one numeric dependent variable and 11 independent variables (3 of them are numeric and the rest are categorical). The number of observations after treating n/a values is 787.

This is the str of my data frame:

'data.frame' : 787 obs. of  12 variables:
 $ a : num  3.02 3.02 3.02 3.02 4.02 4.02 4.02 4.02 4.06 4.06 ...
 $ b : int  300000000 300000000 300000000 300000000 130000000 130000000 130000000 130000000 200000000 200000000 ...
 $ c : Factor w/ 6 levels 
 $ d : Factor w/ 2 levels 
 $ e : Factor w/ 6 levels 
 $ f : Factor w/ 3 levels 
 $ g : Factor w/ 11 levels
 $ h : Factor w/ 4 levels 
 $ i : Factor w/ 2 levels 
 $ k : num  2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 2.9 ...
 $ l : num  74 74 74 74 192 192 192 192 119 119 ...
 $ m : num  183 196 175 311 206 ...

What I want from the model is to rank in a way the independent variables based on their importance. I have some questions with regards to the steps that I need to follow.

Do I have to split my data into train and validation sets? That said is validation needed in the end or is a step that I can skip and thus work with my full data set?

I have run:

library(randomForest)
model1 <- randomForest(y ~ ., data = data_selected, mtry=3, ntree=500, importance = TRUE)
print(model1)

and this is my result:

Call:
randomForest(formula = mid_spread ~ ., data = TrainSet, mtry = 2,      importance = TRUE) 
Type of random forest: regression
Number of trees: 500
No. of variables tried at each split: 2

Mean of squared residuals: 63426.82
% Var explained: 50.72

Don't you think that % Var explained: 50.72 is quite low? I have tried to tune the tree in order to find ideal mtry but improvement is marginal. The same when I try different different number of trees.

I have also used:

importance(model1)        
varImpPlot(model1)

Is this enough? Could anyone give me a suggestion of what steps I need to follow in order to have a complete workflow?

$\endgroup$
  • $\begingroup$ One well-known and principled way to use random forest to find relevant features is called Boruta. You might start there. $\endgroup$ – Sycorax Jan 3 at 18:11
0
$\begingroup$

The random forest object in R contains an importance matrix, which measures the importance of variables as the mean decrease Gini impurity. You can use this Gini importance for ranking, but one should note that the Gini importance has been shown to be biased towards high-cardinality variables (See work by Strobl).

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.