8
$\begingroup$

Multiple Kernel Learning methods aim to construct a kernel model where the kernel is a linear combination of fixed base kernels. Learning the kernel then consists of learning the weighting coefficients for each base kernel, rather than optimising the kernel parameters of a single kernel.

The disadvantages of multiple kernel learning seem to be that they are less interpretable and computationally expensive (as to evaluate the model output you need to evaluate all of the base kernels). So if similar performance can be achieved by simply optimising a single kernel, what are the advantages of MKL?

$\endgroup$
  • $\begingroup$ What does "optimizing parameters of a single kernel" mean? we parameterize the gram matrix of k(x,y)? or of the feature mapping Phi(x)? Is there even a systematic way of doing this? Or is it like, running cross validation on a bunch of kernels then pick the best one? $\endgroup$ – Geralt of Rivia Mar 27 '17 at 13:28
  • $\begingroup$ @GeraltofRivia I just mean tuning the hyper-parameters of a basic kernel function (e.g. the scale parameter of an RBF kernel). An RBF kernel is surprisingly hard to beat in terms of generalisation performance and a weighted sum of kernels (which may include the RBF) seems rather less interpretable, at least to me. $\endgroup$ – Dikran Marsupial Mar 27 '17 at 13:36
13
$\begingroup$

There are two advantages (or rather two use-cases):
1] For every application of SVMs, a user has to choose which kernel to use and sometimes even have to design their own kernel matrices. Is it possible to alleviate choosing kernels or specialized kernel designs? MKL was a step towards that.
2] The second case IMHO is by far a more compelling case. Consider that your data input is a video data + cc. The feature representation of each video consists of video features, audio features and text features. Such a data is known as multi-modal data. Each set of these features may require a different notion of similarity (a different kernel). Instead of building a specialized kernel for such applications, is it possible to just define kernel for each of these modes and linearly combine them?

$\endgroup$
  • 1
    $\begingroup$ +1 However I'm not convinced that [1] is any easier using MKL than just having a linear combination of kernels and choosing the weighting factors via e.g. cross-validation. It also increases the likelihood of over-fitting as there are now more parameters to estimate. As you say, [2] is much more compelling. $\endgroup$ – Dikran Marsupial Jan 11 '13 at 18:50
  • 2
    $\begingroup$ You guys will be interested in this paper by McFee and Lanckriet in JMLR 2011 - jmlr.csail.mit.edu/papers/v12/mcfee11a.html $\endgroup$ – TenaliRaman Jan 11 '13 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.