This is obviously a very general question but most classifiers would use a kernel to map the input into another space where the data could be linearly separated. I also wonder why this is done - I understand that linear separability possibly makes the calculation easier. However if I understand for instance Support Vector Machines correctly, the hyperplane in the kernel space does have a non-linear equivalent in the input space - so why do we not use this non-linear equivalent in the input space from the start? Is the only reason the easier calculation of the kernel function in the kernel space?


1 Answer 1


Of course not. Random forests are known for being capable to separate even very complex classes with nonlinear decision boundary, for which one will think about SVM with some kernel ( http://scikit-learn.org/stable/modules/ensemble.html )

Also ( deep ) autoencoders are not always based on kernel computations but they are nonlinear.

Typically, it's very difficult to find this non-linear equivalent in the input space from the beginning, although, of course, when you know some hidden magic behind you data you can build much simpler classifiers ( and in general you are even obliged to do some investigation of your data prior to solving classification task )

However, to my experience, when you build very strong classifiers for image processing tasks, all really good results are thanks to deep architectures, which are kernel-based under the hood ( and these kernels are learnable! )


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy