CNN vs fully connected network for image recognition? Does anyone know how the accuracy of CNNs compare with fully connected networks for image recognition? Also are CNNs good at anything other than image recognition? I couldn't find anything on Google, a link or explanation would be good. 
 A: Fully connected neural networks are good enough classifiers, however they aren't good for feature extraction. Before the emergence on CNNs the state-of-the-art was to extract explicit features from images and then classify these features.
CNNs are trained to identify and extract the best features from the images for the problem at hand. That is their main strength. The latter layers of a CNN are fully connected because of their strength as a classifier. So these two architectures aren't competing though as you may think as CNNs incorporate FC layers.
If your question was how well a FC-based image recognition technique fairs compared to a CNN one, you should check the results of the ILSVRC for the past years. The last non-CNN architecture I think achieved a top 5 error rate of 30% (today with the state-of-the-art CNNs this is under 3%).
A: 
Does anyone know how the accuracy of CNNs compare with fully connected
  networks for image recognition?

The sole fact that since AlexNet won ImageNet competition, every neural network that wins it uses CNN component, should be enough to convince you that CNNs are better for image data.
You most likely won't be able to find any meaningful comparison, since CNNs are able to handle image data that is infeasible using only FC layers.
Why?


*

*The number of weights in FC layer with 1000 neurons for 224x224x3 image is something like 150M. That's 150M for only one layer. 

*If you're not convinced this is a huge number, then note that modern CNN architectures that have 50-100 layers while having overall couple dozen milion parameters (for example ResNet50 has 23M params, InceptionV3 has 21M parameters).



Also are CNNs good at anything other than image recognition?

Yes. They're good for any data that has spatial structure (for example 1D convolutions over time for music). 
Also they're used in NLP - see this paper on sentiment analysis or this one on translation.
A: 1. Computational tractability.
number of weights between CNN and FC such as
input image of shape 500 x 500 x 3 will be in FC layer with 100 hidden unit (basis = 0)
FC layer = Wx = 100 x ( 500 x 500 x 3 ) = 100 x 750000 = 75M

on other hand:
input image of shape 500 x 500 x 3 will be after convolving a 5 * 5 kernel with zero padding, the stride of 1. and 2 filters
the new CNN layer = ((Hn + 2p - k )/s)+1,((Wn + 2p - k )/s)+1, Cn * filters num)
= 496 x 496 x 6 

the number of parameters in a CONV layer is : ((shape of width of the filter * shape of height of the filter * number of filters in the previous layer+1)*number of filters)
1 for bias
number of parameters = (Fw * Fh * D + 1 ) * F  =  (5 * 5 * 3 + 1 )*2  = 152 

2. Explicit hierarchical representation of features.
the best thing in CNN architecture is no need for feature extraction.
3. Reduces overfitting.
If the model is massively overfitting you can start adding dropout in small pieces. also, max-pooling reduce the overfitting too 
4. Translation invariant.
Invariance refers to the ability to remember an object as an object even though its object place changes. This is usually a positive thing because it maintains the object's identity, category,
"Note that translation here has a specific meaning in vision, borrowed from geometry."

The same object in different locations.
