Why do CNNs perform better on image recognition tasks than fully connected networks?

Question

The way I understand it, each convolution maps regions of an Array to a single entry in a transformed Array repeatedly.

A fully connected layer maps the entire Array to a single entry in the transformed Array repeatedly.

Why is it that image recognition tasks perform better with CNNs? I know that they are more memory efficient and intuitively, learn "features" of an image. But theoretically, doesn't a fully connected network have as much, if not more, capacity as a CNN to learn? What happens in the training process that makes a NN perform worse than a CNN?

Answer 1

I think it is related to the very reason deep models generalize by over-parameterization.

There is a hypothesis that this generalization happens because bigger networks contain more "lottery-ticket" sub-networks during initialization that are close to being "correct" for the objective and backpropagation is good with preserving and utilizing them.

If it is true and training process is deterministic, then we can define an r.v. of scores(function of initialized weights which are r.v.). This r.v.'s distribution will depend on how many "good" sub-networks in our network can appear (and how probable they are). If we know that only spatially related pixels matter in images, then we can use this intuition and limit overall space of sub-networks by ignoring weights that are less probable to be a part of a good solution and therefore increasing a chance of "good" sub-network to appear. That is what CNN's do IMO.

Also as you've mentioned CNN's contain much less connections (because they ignore less necessary ones) and therefore can be bigger/deeper than their fully-connected counterparts.