5
$\begingroup$

I'm currently studying machine learning (support vector machines to be more specific), and I was wondering how exactly I should understand what a kernel function is. I've read other questions on this community such as:

  1. How to intuitively explain what a kernel is?
  2. Understanding kernel functions for SVMs
  3. What function could be a kernel?

However, I'm still having trouble grasping the concept and was hoping somebody would be able to help me out.

My initial understanding is that a kernel is essentially just a mapping into a higher dimension. For example, when we want to make better predictions using a linear classifier, we would use a kernel to map the decision boundary to a higher dimension and make better predictions.

Is my understanding at least on the right track?

Any feedback is appreciated. Thank you.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
9
$\begingroup$

My initial understanding is that a kernel is essentially just a mapping into a higher dimension.

No. Kernel is a function that calculates dot product in the image of this mapping.

It can be thought of as defining dot product, using dot product from another space, where the mapping into this (often higher-dimensional) space is implicit.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the answer. What does it mean for the mapping to be "implicit?" $\endgroup$
    – Sean
    Nov 30, 2018 at 3:54
  • 2
    $\begingroup$ That means that you don't calculate it. Sometimes you even won't be able to calculate it explicitly at all - for example RBF kernel's mapping maps vectors to infinitely-dimensional space. $\endgroup$
    – Jakub Bartczuk
    Nov 30, 2018 at 10:47
  • $\begingroup$ So the kernel is a metric tensor that defines the coordinates, wedge and dot product? $\endgroup$
    – Germania
    Oct 3, 2021 at 5:57
  • $\begingroup$ Not really. A Riemannian metric is a function that given a point on manifold defines dot product. That means m(x)(v,w) is a number, so it can be interpreted as a function of three things that returns a number. Kernel takes two points from one space and returns dot product of their embeddings. That means it's a function of two parameters. $\endgroup$
    – Jakub Bartczuk
    Oct 5, 2021 at 20:13
4
$\begingroup$

My answer may be paraphrasing what you already gathered in the other threads, but here's my way of seeing this.

Technically, the kernel trick can indeed be seen as a mapping to a higher dimension (possibly infinite) where a linear method works. But it is much more than that, because you use an implicit definition of the transformation, through the definition of the inner product in the target space.

$$k(x_i, x_j) = \left< \phi(x_i), \phi(x_j) \right>$$

where $\phi$ is the transformation and $k$ your kernel function.

The kernel function itself can be seen as a measure of similarity between vectors, or (in my interpretation) as a representant for the local behaviour shape. Check out the comparison of how kernels determine the shape of a 1-D gaussian process regressor estimator on this page, I found it very useful on the way to understanding what the kernel choice implies.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.