1
$\begingroup$

I ran randomForest in R package using 7 predictors variables (x1 to x7). I repeated the test with 4 dependent variables (y1 to y4). The importance numbers (IncNodePurity) are plotted in following graph:

enter image description here

Does this mean that the predictors explain maximum variance of y4 than others? How do I know whether predictors are statistically significant or not? What additional test can I run to determine this? Thanks for your help.

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ Boruta could help you: stats.stackexchange.com/search?q=boruta $\endgroup$ – Stéphane Laurent Dec 11 '14 at 19:30
  • $\begingroup$ The sample size is about 8000. Boruta is taking a really long time. $\endgroup$ – rnso Dec 12 '14 at 3:00
  • $\begingroup$ Boruta found all "7 attributes confirmed important". Not very helpful. I check more details on Boruta. $\endgroup$ – rnso Dec 12 '14 at 3:48
  • $\begingroup$ You confuse what is helpful with what suits your wishes... $\endgroup$ – Stéphane Laurent Dec 12 '14 at 6:47
  • $\begingroup$ I guess you are right. Boruta output means that all predictor variables are significant and that is important. What I meant was that it did not show any P values or coefficients or relative importance of different predictors. Maybe I need to use other functions of Boruta for that. But the main problem is that it is much slower than other tests. In my case it took about 27 minutes. $\endgroup$ – rnso Dec 12 '14 at 13:12
3
$\begingroup$

Here is a reference to identification of significant features in RF by Paul et al. http://perso.uclouvain.be/michel.verleysen/papers/ecmlpkdd13jp.pdf which provides p-values. Comparing IncMSE may be more informative in this context than IncNodePurity, but exact values of both are often unstable, and the solution depends on the total amount of variation explained (interestingly, the graph shows that predictors rank rather consistently across all y's). Also see this discussion Are randomForest variable importance values comparable across same variables on different dates?

Improve this answer
$\endgroup$
3
  • $\begingroup$ What can we infer from "predictors rank rather consistently across all y's"? Does it mean that all the y's have same predictors and hence may be related to each other? And what can we infer from taller y4 bar than y3 bar. $\endgroup$ – rnso Dec 11 '14 at 17:47
  • $\begingroup$ Also, see: stats.stackexchange.com/questions/127597/…. I am surprised that the predictors identified as most important by randomforest were not included in bestglm model. $\endgroup$ – rnso Dec 11 '14 at 17:50
  • $\begingroup$ It is plausible for the two or more variables to have the same predictors yet be unrelated to each other. $\endgroup$ – katya Dec 11 '14 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.