# Mixed SVM kernel of RBF and linear

I've read some introduction about different kernels for SVM. It seems RBF is a measure of point distance while the basic kernel (i.e. no kernel) splits the space by hyper-planes.

I could imagine that for a mix of features, some features should be treated with RBF and some with the basic kernnel.

Is it possible use RBF for some features and the basic vector product for the other features?

-
I'd look into something called multiple kernel learning. –  gamerx Mar 23 '13 at 7:35

What you're describing would send your gram matrix to a much larger feature space than a single kernel. Consider the following. Given two valid mercer kernels, $\alpha_1K_1(x_i,x_j) + \alpha_2K_2(x_i, x_j)$ is a valid kernel for all $\alpha_1,\alpha_2\in\mathbb R^+$. Similarly $K_1(x_i,x_j)K_2(x_i,x_j)$ is also a valid kernel. These results imply that arbitrary polynomial expansion can be applied to kernels, allowing for interaction between kernels. You can get a the desired result, at the normal speed of evaluating a gram matrix.

-

That's a really interesting idea! I've done some work with ensemble classifiers that treats different samples with a particular classifier, depending on the confidence level of each, but never at the feature level. Off the top of my head, I think it makes the most since to just do what you're describing at the classification level--classify each sample with both classifiers, and accept the answer of the most-confident one. I'm not sure how it would work at the feature level, as you describe, but I'd be interested to hear your thoughts!

Do you use a standard classification package (e.g., Weka), or have you coded your own pipeline? In my experience, these sorts of outside the box ideas are much easier to do with a system you've coded from the ground up. It gives you a much better grasp of the plumbing connecting the different components of the framework.

-

Given:

1. SVMs are similar conceptually to regression modelling

2. regression models can be fit with a mixture of locally-weighted kernels and linear features (they're called generalised additive models)

I'd say sure, go for it.

-