1
$\begingroup$

When reading about SVM and Kernel Trick, a common similarity function which is often used is enter image description here

which is taken from this blog

I am trying to understand how this is related to equation of circle. Since this Kernel is being used to separate data which is distributed like a circle.

enter image description here

Following the complete math which is not in this blog but explained in this video, this is equivalent to square of dot products in lower dimension $(X^TY)^2$. This is explained in this video tutorial in detail (around 3:00). However, I do not get it how he arrived from $(X^TY)^2$ to equation of circle.

$\endgroup$

1 Answer 1

2
$\begingroup$

An SVM learns the parameters $a$, $b$, $c$ and $d$ of a separating hyperplane in the new space $\phi(\mathbf{x}) = \phi(\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}) = \begin{pmatrix} x_1^2 \\ \sqrt{2}x_1x_2 \\ x_2^2 \end{pmatrix} = \begin{pmatrix}X\\Y\\Z\end{pmatrix}$, which has an equation of the form $aX+bY+cZ=d$. Take the particular case where $a=c=1$ and $b=0$, you obtain (with variables from the input space) the equation of a circle of radius $\sqrt{d}$ and center $(0, 0)$: $x_1^2 + x_2^2=d$. According to your figure, choose $d\approx0.7^2$, and this hyperplane should separate your data. Note that the SVM will probably not learn exactly these particular parameters ($a=c=1$, $b=0$, $d=0.7^2$). Based on the form of the kernel space, the SVM separating plane could be ellipses, hyperbolas or parabolas (see: https://en.wikipedia.org/wiki/Conic_section#General_Cartesian_form). However the SVM will learn the curve that separates the data with the larger margin. So probably that the result of your SVM should be an ellipse close to a perfect circle (with the center exactly in $(0, 0)$ because you don't have terms in just $x_1$ or $x_2$ in the equation of the ellipse, and according to this page: https://en.wikipedia.org/wiki/Ellipse#General_ellipse), and with radius between $0.6$ and $0.8$ or something like that (this is just by looking approximately on your figure).

$\endgroup$
1
  • $\begingroup$ Makes sense. General cartesian form of conics was the link I was missing and now it is perfectly clear! $\endgroup$
    – PagMax
    Commented Oct 18, 2019 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.