# How to use the kernel trick on data that can't be visualized

I'm reviewing the kernel trick and there are a lot of toy examples of how a 2D classification which can't usually be separated by a linear SVM can be separated in 3 space.

This is fine, but how is the kernel trick applied in a real environment where you have maybe hundreds of features? The immediate problem I see is that you can't visualize that many dimensions. So how would one even know what, if any, kernel needs to be used?

Off the top of my head, it seems like if you did dimensionality reduction first to get down to 3D, then maybe one would have an easier time applying the kernel trick.

• – Marc Claesen Mar 26 '15 at 6:08

Which kernel to use depends on the problem. In general the RBF kernel tends to work well: $$\kappa(\mathbf{u},\mathbf{v}) = \exp(-\gamma \|\mathbf{u}-\mathbf{v}\|^2).$$ When using this kernel, you still have to find a suitable value for $\gamma \in \mathbb{R}^+_0$, which is typically part of the hyperparameter optimization. A specific value of $\gamma$ can be evaluated via techniques like cross-validation.