I understand the convolutional and pooling layers, but I cannot see the reason for a fully connected layer in CNNs. Why isn't the previous layer directly connected to the output layer?


2 Answers 2


The output from the convolutional layers represents high-level features in the data. While that output could be flattened and connected to the output layer, adding a fully-connected layer is a (usually) cheap way of learning non-linear combinations of these features.

Essentially the convolutional layers are providing a meaningful, low-dimensional, and somewhat invariant feature space, and the fully-connected layer is learning a (possibly non-linear) function in that space.

NOTE: It is trivial to convert from FC layers to Conv layers. Converting these top FC layers to Conv layers can be helpful as this page describes.

  • $\begingroup$ Thanks for your answer James. So we are learning the weights between the connected layers with back propagation, is it correct? $\endgroup$
    – jeff
    Nov 17, 2015 at 3:15
  • $\begingroup$ Yes the error back-propagates through the fully-connected layer to the convolutional and pooling layers. $\endgroup$
    – jamesmf
    Nov 17, 2015 at 3:17
  • 1
    $\begingroup$ Ok. So the purpose of the f.c. layer can be thought like non-linear PCA, it rectifies the "good" features and diminishes the others via learning the full set of weights. $\endgroup$
    – jeff
    Nov 17, 2015 at 3:25
  • 2
    $\begingroup$ It mostly allows you non-linear combination of features. All the features may be good (assuming you don't have "dead" features), but combinations of those features might be even better. $\endgroup$
    – jamesmf
    Nov 17, 2015 at 3:30
  • $\begingroup$ @jamesmf: What is a dead feature? and what are combinations of features you are talking about? what do you mean by a non linear combination? Is using a fully connected layer mandatory in a cnn? or can it be substituted without any adverse effect on accuracy? Thanks alot in advance. I;d be grateful if you could give an intuition on the questions I asked. $\endgroup$
    – Hossein
    May 17, 2016 at 7:47

I found this answer by Anil-Sharma on Quora helpful.

We can divide the whole network (for classification) into two parts:

  • Feature extraction: In the conventional classification algorithms, like SVMs, we used to extract features from the data to make the classification work. The convolutional layers are serving the same purpose of feature extraction. CNNs capture better representation of data and hence we don’t need to do feature engineering.

  • Classification: After feature extraction we need to classify the data into various classes, this can be done using a fully connected (FC) neural network. In place of fully connected layers, we can also use a conventional classifier like SVM. But we generally end up adding FC layers to make the model end-to-end trainable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.