# How are SVMs = Template Matching?

I read about SVMs and learnt that they are solving an optimization problem and max margin idea was very reasonable.

Now, using kernels they can find even non-linear separation boundaries which was great.

So far, I really do not have any idea how SVMs (a special kernel machine) and kernel machines are related to neural networks?

Consider the comments by Yann Lecun => here:

kernel methods were a form of glorified template matching


and here too:

For example, some people were dazzled by kernel methods because of the cute math that goes with it. But, as I’ve said in the past, in the end, kernel machines are shallow networks that perform “glorified template matching”. There is nothing wrong with that (SVM is a great method), but it has dire limitations that we should all be aware of.

So my questions are:

1. How is SVM related to neural network? How is it a shallow network?
2. SVM solves an optimization problem with a well defined objective function, how is it doing template matching? What is the template here to which an input is matched?

I guess these comments need a thorough understanding of high dimensional spaces, neural nets and kernel machines but so far I have been trying and couldn't grasp the logic behind it. But it is surely interesting to note the connections between two very very different ml techniques.

EDIT: I think understanding SVMs from a Neural perspective would be great. I am looking for a thorough mathematics backed answer to the above two questions, so as to really understand the link between SVMs and Neural Nets, both in the case of linear SVM and SVMs with the kernel trick.

• SVMs are fairly easy & fast to train given an appropriate kernel. Some tasks don't need deep neural net. – Vladislavs Dovgalecs Feb 23 '17 at 17:23
• @xeon hi, can you take a look at the answer, I suppose it needs improvement. thanks. – Rafael Mar 3 '17 at 7:32