Using Random Forest Variable Importance to train SVM models (R)

Question

I have trained a Random Forest model in R with the caret package but the results are not very promising. I have decided to try with SVM models but I have a great dilemma:

Would it be acceptable to use the "ranking" of important variables given by the Random Forest model (varImp function) to train SVM models with different number of predictors (based on this ranking)?

I supose that this Variable Importance is specific to the Random Forest model but I would like to know if it would make sense to apply this ranking to other models like SVM.

Answer 1

Even though the varImp is calculated individually by each model (model type), the underlying information should to some extent generalize across models. For example, a random variable should be labeled as unimportant by all model types, while e.g. a single, very distinctive feature should be recognized as such by all model types.

One can come up with situations where this is not possible: e.g. when trying to model an XOR-like-problem using a linear model (underfitting), the variable importance will be less useful. But in case the model is powerful enough I'd guess that the derived variable importance captures some general, model independent about variable importance too. And this should be true especially with trees (therefore to some extent with tree based models like forests too), where more important variable tend to be closer to the root node.

Having said this: I'd try to use the RF varImp just from curiosity, but I'd also try not using it and see for the differences, or derive new variable importance from your other model types and try to compare them (e.g. ranking).