How are convolutional networks better than simple neural networks (Feed-forward networks)? Is the convolution operation the only difference between convolutional neural networks and deep networks?

Any time that you can legitimately make stronger assumptions, you can obtain stronger results. Convolutional networks make the assumption of locality, and hence are more powerful. This depends on data that in fact exhibits locality (autocorrelation) like images or time series.

Intuitively, if you are looking at an image, pixels in a region of the image are more likely to be related than pixels far away. So you can save a lot of neuron wiring if you don't (directly) wire distant pixels to the same neuron. With less wiring, you have more data per coefficient, which speeds things up and makes for better results.

Deep nets are a general class whose solely mandatory characteristic is an unusual number of neural layers, whereas convolutional networks are a specific technique that can be included in a deep net, alongside other techniques such as LSTMs, perceptrons, Kohonen nets, etc. Keep in mind that "deep learning" is something of a buzz word with a fuzzy definition; in practice, these other neural techniques aren't included anywhere near as often in the equally fuzzy definition of deep nets. I believe that most "deep" networks consist mainly of convolutional layers, based on the academic papers I've read so far. The term doesn't necessarily imply or demand the inclusion of convolutionals in every deep network - you could, for example, have a deep net consisting of two dozen perceptron layers, although that would hardly be practical. There are practical advantages, however, to piling convolutionals on top of each other in stacks, particularly the ability to leverage convolution filters to create hierarchical representations. There may be other subtle advantages to convolutionals, but that is by far the dominant one. I'll leave it to more experienced users to provide the mathematical and logical details as to why convolutionals can be stacked to create hierarhical representations, but suffice it to say that this is the "short form" answer to your questions: 1) they're better mainly because of the convolutional filtering operation and 2) is something of an apples and oranges comparison, since one is a general class and the other is a specific type often included within it.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.