# How to choose the best algorithm for measuring attribute importance/relevance?

Let's say we want to conclude that attributes A, B, C and D are the most relevant attributes to maximize the precision of predicting Y, and then rank those attributes based on their importance/relevance.

Now let's say SVM and Random Forest both seem to be good fits to model the data and Random Forest provides better performance (higher precision). Does that mean Random Forest would also be a better choice to rank the attributes according to their relevance to D?

In a more general sense, can we say the best algorithm for maximizing precision of predicting Y is also the best algorithm to rank the relevance of attributes to Y?

• All variable importance measures tell you only how a specific algorithm used your data. When you report variable importance, you should keep this in mind: "This is a summary of how the algorithm I chose used my training data". There is no concept of variable importance that is algorithm independent that is being measured. Consequently, the variable importance measures you get from different algorithms are not comparable to eachother. – Matthew Drury Dec 13 '16 at 19:13
• @MatthewDrury then if the goal is to rank attribute relevance, could somehow averaging the weight of multiple algorithms be a good approach? Like if both SVM and RF are saying A is the most relevant, could we say A is potentially the most relevant? How do you compare this approach to a simple correlation test between A and Y? – Aliweb Dec 13 '16 at 19:17
• As the poster of this question: stats.stackexchange.com/questions/202277/… I'm not sure I'm the best person to help you. I've stopped believing that it's even possible to rank the influence of predictors in a model independent way. – Matthew Drury Dec 13 '16 at 19:52