2
$\begingroup$

The linear SVM function can be expressed as:

$$h(x)=w^{T}x+w_{0}$$

where $x=({{x}}_{1},\dots,{{x}}_{N})^{T}$ is one observation and $w=({{w}}_{1},\dots,{{w}}_{N})^{T}$ are the coefficients.

We can do the calculation of the dot product in the following way:

$$w^{T}x=x_1w_1+\dots+x_Nw_N$$

Now with the kernel trick, the function $h$ is expressed as:

$$h(x)=w^{T}\phi (x)+w_{0}$$

And my question is : for a given observation $x$, how the calculation is done ? since, $\phi$ is always expressed as:

$$K(x_{i},x_{j})=\phi (x_{i})^{T}\cdot \phi (x_{j})$$

With some example of $K$

  • Polynomial kernel : $K(x_{i},x_{j})=(x_{i}^{T}\cdot x_{j}+1)^{d} $
  • Gaussian kernel: $ {\displaystyle K(\mathbf {x} ,\mathbf {y} )=\exp \left(-{\frac {\|\mathbf {x} -\mathbf {y} \|^{2}}{2\sigma ^{2}}}\right)}$

PS: with the dual form, we can express $h$ as:

$$h(x)=\sum _{{k=1}}^{p}\alpha _{k}^{*}l_{k}K(x_{k},x)+w_{0}$$

But I try to understand how to write the function $\phi$ in an explicit way.

$\endgroup$
4
  • 1
    $\begingroup$ The expression you're asking about ($h(x) = w^T \phi(x) + w_0$) does not use the kernel trick. The point of the kernel trick is to avoid explicit computation of the feature map $\phi$, which may map to a very high (or possibly infinite) dimensional space. The 'dual form' expression you wrote for $h(x)$ is how the kernelized version should look. $\endgroup$
    – user20160
    Dec 29, 2021 at 14:34
  • $\begingroup$ @user20160 This seems like an answer to the question. Would you care to write it up? $\endgroup$
    – Sycorax
    Dec 29, 2021 at 17:17
  • $\begingroup$ Thank you for your comment, I found this post useful to understand that apply $\phi$ is not the kernel trick. But then I don't know how to say it exactly, maybe we should say: apply $\phi$ to project x to a feature map. Which one ? if it is polynomial, do we say polynomial feature map ? or feature map with polynomial kernel ? for the latter, we use "kernel"... $\endgroup$
    – John Smith
    Dec 30, 2021 at 15:40
  • $\begingroup$ To express explicit $\phi$ for a gaussian kernel (notice the word kernel is used), this post gives the answer. And for a polynomial kernel, the wikipedia article gives the answer. $\endgroup$
    – John Smith
    Dec 30, 2021 at 15:46

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.