Using a gaussian kernel in SVM. How exactly is this then written as a dot product? I am attempting to use SVMs for my class project. For this project, I have selected the gaussian kernel as, well, the kernel. That is,
$$
k(\mathbf{x}_1, \mathbf{x}_n) = e^{-\gamma ||\mathbf{x}_1 - \mathbf{x}_n ||^2}
$$
What I do not understand, is how this kernel is then 'written as a dot-product'. How do we get around doing that? This is because my professor says that when we finalize the training, we will be performing a dot-product between a new vector and the SVs. But given this kernel, how is this dot-product being done?
 A: Look up "kernel trick". The idea is that, under certain conditions (Mercer's condition), a function $k(x,x')$ can be expressed as a dot product $<\phi(x),~\phi(x')>$, where $\phi$ is a function that transforms $x$ into a high dimensional (possibly infinite) representation.
The trick is that, as long as your optimization problem can be expressed solely with dot products, you do not need to know or compute $\phi$, you simply use the kernel function $k$. 
More details on Wikipedia
A: Once you derive this term:
$$
k(\vec x, \vec y) = e^{-\gamma ||\vec x - \vec y ||^2}
$$
you get terms that only depend on $\vec x$, terms that only depend on $\vec y$, and terms that are mixed.
The terms that separate nicely, aren't a problem, as you could write them as a dot product.
e.g. in the simple case where x is 1-D:
$$
k(\vec x, \vec y) = e^{-\gamma ||x - y ||^2} = e^{-\gamma (x^2 -2xy + y^2)} \\ 
e^{-\gamma (x^2 + y^2)} = e^{-\gamma (x^2)} e^{-\gamma (y^2)}
$$
Here you see the separate terms can be separated into a (dot) product.
The terms that are mixed are a problem - but the trick is to use Taylor series to separate them into an infinite series of products:
$$
e^{2\gamma xy} = \sum_{k = 0}^{\infty} \frac{(2\gamma xy)^k}{k!} 
$$
Here $x, y$ are products.
So the gaussian kernel can be written as the dot product between the following vectors:
$$
e^{-\gamma (x^2)}<1, \sqrt{2 \gamma}x, \frac{2 \gamma x^2}{\sqrt 2}, ...> \cdot e^{-\gamma (y^2)}<1, \sqrt{2 \gamma}y, \frac{2 \gamma y^2}{\sqrt 2}, ...>
$$
