This is obviously a very general question but most classifiers would use a kernel to map the input into another space where the data could be linearly separated. I also wonder why this is done - I understand that linear separability possibly makes the calculation easier. However if I understand for instance Support Vector Machines correctly, the hyperplane in the kernel space does have a non-linear equivalent in the input space - so why do we not use this non-linear equivalent in the input space from the start? Is the only reason the easier calculation of the kernel function in the kernel space?


Of course not. Random forests are known for being capable to separate even very complex classes with nonlinear decision boundary, for which one will think about SVM with some kernel ( http://scikit-learn.org/stable/modules/ensemble.html )

Also ( deep ) autoencoders are not always based on kernel computations but they are nonlinear.

Typically, it's very difficult to find this non-linear equivalent in the input space from the beginning, although, of course, when you know some hidden magic behind you data you can build much simpler classifiers ( and in general you are even obliged to do some investigation of your data prior to solving classification task )

However, to my experience, when you build very strong classifiers for image processing tasks, all really good results are thanks to deep architectures, which are kernel-based under the hood ( and these kernels are learnable! )


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.