1
$\begingroup$

I am new in using RF. I want to use it to compute the relative importance of the features. I found the weight is very small ("party" package, cforest). Is there anyway to get these weights in a range of 0-1? The total weight would sum up to 1? For example, if $x_1$, $x_2$ and $x_3$ are the features, the relative importance for these features are something like, $r_1: 0.5, r_2:0.4$ and $r_3:0.1$.

Thanks in advance.

$\endgroup$

1 Answer 1

1
$\begingroup$

You can normalize them if you really want - I don't see any problem with it.

But keep in mind that the importance measure is constructed this way intentionally. If you have two variables and each results in 0.7 importance, it doesn't mean they put in 50% of influence each. They can be strongly correlated (or even be the same variable altogether). The idea of the measure is actually how different is your variable's importance from a random variable. So if it's close to 0 it means that it gives almost no information about the target.

$\endgroup$
2
  • $\begingroup$ Thank you very much sashkello. Now I understand, before I thought that its relative to each other. Is there any way to roughly estimate such thing as I mentioned in my question? By the way, I couldn't vote your answer (not enough reputation!). Actually, I wanted to develop an equation where each feature would contributes to a score. Their contribution would be higher if their importance is higher! Any method that can absorb multicollinearity issue too? Thanks. $\endgroup$ Apr 10, 2013 at 14:28
  • $\begingroup$ In terms of dealing with collinearity, I am not aware of any tool for random forest which can improve on that. The importance measure (sort of) tells you how much % of the output this variable can predict. In such terms one variable is no better than other, you can't separate them. There are many ways to deal with collinearity, but I don't know about anything specific to random forests unfortunately... $\endgroup$
    – sashkello
    Apr 10, 2013 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.