3
$\begingroup$

According to the book "Support Vector Machines" from Cristianini and Shawe-Taylor, it is feasible to make kernels from kernels.

My question is now more in application of this methods with tools like R.

In particular, I have two models with the same classes, but the data is different.

Do you know how I can combine two SVM kernels, i.e. linear and radial, to one learner for the classification task? Can you provide me some useful documents or links to libraries to do so?

$\endgroup$
2
  • 6
    $\begingroup$ You can simply add the kernels together, or multiply them, or do other things. (This is established in many sources, including the one you mention; I recently wrote a few such properties with proofs here.) Is your question how to do this in code? This varies depending on the language and SVM package; with the popular libsvm package, you need to give it a precomputed kernel matrix, so just explicitly form both kernel matrices and add them together. $\endgroup$ – Danica May 15 '15 at 7:47
  • $\begingroup$ Adding kernels is one of the arguably simplest. You can weight each kernel automatically using Multiple Kernel Learning approach. Note that kernel sum is merely concatenation of the two feature space vectors. $\endgroup$ – Vladislavs Dovgalecs May 26 '15 at 18:20
4
$\begingroup$

As Dougal comments, you can just add, average or multiply the values of individual kernels. By way of example, the authors of this paper (1) propose methods for combining kernels in the context of ordinal, categorical and continuous-valued data found in clinical medicine. Their solution involves computing a kernel function for each feature, with the function to be computed depending on the type of data involved: they have an ordinal kernel, a categorical kernel and a continuous-valued kernel. Then they scale each of the function values to the $[0,1]$ interval and average the several kernel matrices to produce their final kernel matrix.

To your specific question about how to combine radial and linear kernels, you should consider what properties you want the kernel function to have. The only constraint on kernel functions is that they must be symmetric, positive semi-definite, but within those very general constraints, you can make manipulations of the radial and linear kernel functions' values to emphasize some attributes of your data, or de-emphasize others.

As for achieving a combination of kernel functions with software, that's a programming problem, rather than a statistical one... But in R, supposing that you want to average two kernel matrices $A$ and $B$ of the same dimension, you can use something like

C <- (A+B)/2

and the result is also a square kernel matrix of the same dimension as $A$ and $B$.

  1. Anneleen Daemen and Bart De Moor, "Development of a Kernel Function for Clinical Data," 31st Annual International Conference of the IEEE (2009)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.