I was revising Convolutional Neural Networks and encountered the following question. If I were to classify a cat and a dog (the famous cats vs dog classifier), then assuming there are 2 Dense Layers after the CNN layers, and of course 1 output layer, the question is if the 2nd Dense Layer Neurons can have the same neurons as the first Dense Layer.

More concretely, if Dense 1 has 3 Neurons (feature of the neuron = color, head, tail) and Dense 2 has 2 neurons, can the feature of the Neuron be color and head? Logically it should be okay, since Dense Layers serve as Feature Selector, whereby it selects the most relevant features.


Second dense layer gets the output of the first dense layer, so the features are not the same any more, but transformed versions of the initial ones.

Dense Layers serve as Feature Selector...

They're not feature selectors, but transformers such that the most expressive features are selected in the end for the final layer. But, those features may lose interpretability after so many transformations.

You can however also input the initial features in addition to the first layer outputs, to the second dense layer. This structure is called residual NNs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.