Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I've read some introduction about different kernels for SVM. It seems RBF is a measure of point distance while the basic kernel (i.e. no kernel) splits the space by hyper-planes.

I could imagine that for a mix of features, some features should be treated with RBF and some with the basic kernnel.

Is it possible use RBF for some features and the basic vector product for the other features?

share|improve this question
    
I'd look into something called multiple kernel learning. –  gamerx Mar 23 '13 at 7:35
add comment

3 Answers

up vote 1 down vote accepted

What you're describing would send your gram matrix to a much larger feature space than a single kernel. Consider the following. Given two valid mercer kernels, $\alpha_1K_1(x_i,x_j) + \alpha_2K_2(x_i, x_j)$ is a valid kernel for all $\alpha_1,\alpha_2\in\mathbb R^+$. Similarly $K_1(x_i,x_j)K_2(x_i,x_j)$ is also a valid kernel. These results imply that arbitrary polynomial expansion can be applied to kernels, allowing for interaction between kernels. You can get a the desired result, at the normal speed of evaluating a gram matrix.

See: Classes of Kernels for Machine Learning: A Statistics Perspective

share|improve this answer
add comment

That's a really interesting idea! I've done some work with ensemble classifiers that treats different samples with a particular classifier, depending on the confidence level of each, but never at the feature level. Off the top of my head, I think it makes the most since to just do what you're describing at the classification level--classify each sample with both classifiers, and accept the answer of the most-confident one. I'm not sure how it would work at the feature level, as you describe, but I'd be interested to hear your thoughts!

Do you use a standard classification package (e.g., Weka), or have you coded your own pipeline? In my experience, these sorts of outside the box ideas are much easier to do with a system you've coded from the ground up. It gives you a much better grasp of the plumbing connecting the different components of the framework.

share|improve this answer
add comment

Given:

  1. SVMs are similar conceptually to regression modelling

  2. regression models can be fit with a mixture of locally-weighted kernels and linear features (they're called generalised additive models)

I'd say sure, go for it.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.