2
$\begingroup$

I'm currently studying machine learning (support vector machines to be more specific), and I was wondering how exactly I should understand what a kernel function is. I've read other questions on this community such as:

"How to intuitively explain what a kernel is?",

"Understanding kernel functions for SVMs", and

"What function could be a kernel?".

However, I'm still having trouble grasping the concept and was hoping somebody would be able to help me out.

My initial understanding is that a kernel is essentially just a mapping into a higher dimension. For example, when we want to make better predictions using a linear classifier, we would use a kernel to map the decision boundary to a higher dimension and make better predictions.

Is my understanding at least on the right track?

Any feedback is appreciated. Thank you.

$\endgroup$
7
$\begingroup$

My initial understanding is that a kernel is essentially just a mapping into a higher dimension.

No. Kernel is a function that calculates dot product in the image of this mapping.

It can be thought of as defining dot product, using dot product from another space, where the mapping into this (often higher-dimensional) space is implicit.

$\endgroup$
  • $\begingroup$ Thanks for the answer. What does it mean for the mapping to be "implicit?" $\endgroup$ – Seankala Nov 30 '18 at 3:54
  • 2
    $\begingroup$ That means that you don't calculate it. Sometimes you even won't be able to calculate it explicitly at all - for example RBF kernel's mapping maps vectors to infinitely-dimensional space. $\endgroup$ – Jakub Bartczuk Nov 30 '18 at 10:47
2
$\begingroup$

My answer may be paraphrasing what you already gathered in the other threads, but here's my way of seeing this.

Technically, the kernel trick can indeed be seen as a mapping to a higher dimension (possibly infinite) where a linear method works. But it is much more than that, because you use an implicit definition of the transformation, through the definition of the inner product in the target space.

$$k(x_i, x_j) = \left< \phi(x_i), \phi(x_j) \right>$$

where $\phi$ is the transformation and $k$ your kernel function.

The kernel function itself can be seen as a measure of similarity between vectors, or (in my interpretation) as a representant for the local behaviour shape. Check out the comparison of how kernels determine the shape of a 1-D gaussian process regressor estimator on this page, I found it very useful on the way to understanding what the kernel choice implies.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.