Can someone explain the difference between variables of importance from random forest vs all-relevant features from Boruta feature selection?
For example, if one were to build a model (could be any model) using a sub-set of 'important' or 'relevant'features, would it be better to use the output from Boruta all-relevant feature selection, or the Random Forest 'variable of importance'? Is one method preferred over the other? If so why?
Boruta algorithm uses randomization on top of results obtained from variable importance obtained from random forest to determine the truly important and statistically valid results. For details of the difference please refer to Section 2 of the article:
Kursa, Miron B., and Witold R. Rudnicki. "Feature selection with the Boruta package." (2010).
This is a classic case of "No Free Lunch" theorem. Without data and assumptions, it is impossible to decide which one is better. However, please note Boruta is produced as an improvement over random forest variable importance. So, it should perform better in more situations than not (Biased because I like randomization techniques myself). Nevertheless, data and computational time could make variable importance from random forest a better choice.
Perform shuffling of predictors' values and join them with the original predictors and then build random forest on the merged dataset. Then make comparison of original variables with the randomised variables to measure variable importance. Only variables having higher importance than that of the randomised variables are considered important.
In random forest, the Z score is computed by dividing the average accuracy loss by its standard deviation. It is used as the importance measure for all the variables. But we cannot use Z Score which is calculated in random forest, as a measure for finding variable importance as this Z score is not directly related to the statistical significance of the variable importance. To workaround this problem, boruta package runs random forest on both original and random attributes and compute the importance of all variables. Since the whole process is dependent on permuted copies, we repeat random permutation procedure to get statistically robust results.
Answer is NO. You need to test other algorithms. It is not possible to judge the best algorithm without knowing data and assumptions. Since it is an improvement on random forest variable importance measure, it should work well on most of the times.
Check out the original article - Feature selection with Boruta in R to see implementation of Boruta Algorithm with R and its comparison with other feature selection algorithms.
