0
$\begingroup$

So far as I understood kernels, they compute a dot product of the vectors

$k(x_i,x_j) = \langle\phi(x_i),\phi(x_j)\rangle$

where $\phi$ goes from original lower dimensional space to a higher dimensional space. The most popular ones are the polynomial Kernel, the radial basis function kernel and the sigmoid kernel.

Regarding this, I have three questions:

  1. For each of these kernels, what is actually the function $\phi$ look like (in dependence of the hyperparameters)?

  2. Assuming there is a nonlinear decision boundary, that seperates the dataset perfectly with given features, does there always exist a kernel, so that we find this decision boundary with this kernel and SVM?

  3. What would be an example of an Dataset (which actually has a nonlinear decision boundary, sperating this set perfectly), where one of those 3 popular kernels fails?

$\endgroup$
  • 1
    $\begingroup$ For 2&3, what if $x_i = x_j$ but $y_i \neq y_j$? In this case, it's impossible to find a boundary to separate the two classes with any kernel. $\endgroup$ – Sycorax Dec 21 '16 at 16:23
  • $\begingroup$ Ok, excluding this special case :D $\endgroup$ – Luca Thiede Dec 21 '16 at 16:26
  • 1
    $\begingroup$ "Are there any black swans?" "How about this one." "I mean except for that one." $\endgroup$ – Sycorax Dec 21 '16 at 17:15
  • $\begingroup$ Than let me reformulate my question: Assuming there is a nonlinear decision boundary, that seperates the dataset perfectly with given features, does there always exist a kernel, so that we find this decision boundary with this kernel and SVM? $\endgroup$ – Luca Thiede Dec 22 '16 at 9:35
  • $\begingroup$ Same for 3. just considering datasets, where a perfect (perfect under the given dataset) decision boundary actually exists $\endgroup$ – Luca Thiede Dec 22 '16 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.