Idea of the kernel trick I'm reading this article and I can't really grasp the idea of this so-called kernel trick.
So far, what is present, is:
$ \Phi(x)^T * \Phi(y) = \sum x_ix_jy_iy_j$ and
$ k(x, y) = (x^T*y)^2 = \sum x_ix_jy_iy_j $
I don't see the difference. The "trick" is probably, that those equations are the same(?), so it is sufficient to calculate the lower one but I wonder where is the "trick", where is the computational saved effort as I would calculate the sum anyway, in both cases?
 A: The example considers a particular feature mapping, $\Phi(x)$, which is 9 dimensional. Its dot product contains $9$ multiplications. In the trick version, you have $3$ multiplications (i.e. dot product of three dimensional vecs: $x^Ty$), and a squaring operation. There comes the computational efficiency.
Sometimes, the feature dimension can even be infinite, just like in RBF kernel. In such cases, the computational efficiency is certain.
A: Yes, the trick is that both equations compute the exact same thing. A kernel effectively computes the distance between two objects in some high-dimensional space, and the trick is that you never actually need to put anything into the high-dimensional space.
Mapping things into high-dimensional space can be expensive. The naive approach to finding distance in the high-dimensional space is to map your samples to that space, and then find the distance between those two objects in that space. The kernel trick allows you to never actually compute the representation of any sample in the high-dimensional space, you can just compute the distance between samples directly from the original features. The "trick" is that you never need to compute the kernel representation of any sample $\Phi(x)$, which can be costly.
