The way I understand it, each convolution maps regions of an Array to a single entry in a transformed Array repeatedly.
A fully connected layer maps the entire Array to a single entry in the transformed Array repeatedly.
Why is it that image recognition tasks perform better with CNNs? I know that they are more memory efficient and intuitively, learn "features" of an image. But theoretically, doesn't a fully connected network have as much, if not more, capacity as a CNN to learn? What happens in the training process that makes a NN perform worse than a CNN?