Is there a way to determine the important features (weight) for an SVM that uses an RBF kernel?

Question

Using sklearn, I did both a linear kernel SVM and a rbf one. While the rbf gave really great results, I can't determine the important features that the algorithm kept (or used more).

I know that "coef_" does only work for a linear kernel, since for rbf the data space is no longer finite (or at least, it changes [I think]).

Is there a way to actually determine the important features for an RBF kernel SVM ?

Thanks,

Answer 1

Unfortunately not. Although SVMs are often interpreted as transforming your features into a high-dimensional space and fitting a linear classifier in the new space, the transformation is implicit and cannot be easily retrieved.

In fact, SVMs with the RBF kernel behave more like soft nearest neighbours. To see this, denote by $\{x_i,y_i\}_{i=1}^N$ the training data, and $K(.,.)$ the kernel of choice: in this case $K(x,x')=\exp(-\gamma\|x-x'\|^2)$. Then the SVM prediction for an example $x$ takes the form $$ \mathrm{sign}\left( \sum_{i=1}^N \alpha_i y_i K(x_i, x) + \rho \right), $$ where $\rho$ is the intercept_ and $\alpha_i$ are the dual_coef_ in sklearn (see here). As you can see, the decision function is just a linear combination of training labels $y_i$, where the influence of each training example $x_i$ is determined by its overall importance $\alpha_i$ and its distance from $x$, as given by $K$. Closer points have an exponentially larger effect on the prediction, hence the nearest neighbour analogy.

Coming back to your question: maybe it doesn't make so much sense to think in terms of features for SVMs with RBFs (would you ask about features for nearest neighbours?). You may want to have a look at the most influential data points, though, to get a "template" interpretation of your model.