2
$\begingroup$

I am using scikit-learn to build two binary classification models--one is a random forest, and the other is a linear SVM. I want to compare the relative importances of the input features to the trained models.

LinearSVC has an attribute "coef_" while the random forest classifier has an attribute "feature_importances_". I know that they are not computed on the same scale but, from what I've read, both represent the learned importances of the features.

If I transform the coefficients and feature importances to the same scale, would it be appropriate to compare the relative importances/weights assigned to each feature to understand how the two models differ in learning the data?

$\endgroup$
1
  • $\begingroup$ No, because random forest importance metrics are based on out-of-bag observations and there is no such thing in the development of SVM. $\endgroup$
    – stans
    Jul 31, 2023 at 22:52

1 Answer 1

0
$\begingroup$

This is probably a bit of an opinion-based question, so FWIW here's my opinion.

I think it's fine to compare them qualitatively ('feature A is more important than feature B', 'the models agree that feature C is noise', etc).

And I think rescaling them for a quantitative comparison is an interesting idea, but I am not aware of any theory supporting it or suggesting how to do it. Perhaps softmax scaling could work? I think it's a research topic.

By the way, you might want to check out sklearn.inspection.permutation_importance() too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.