3
$\begingroup$

Note that the number of filters grows as we climb up the CNN toward the output layer (it is initially 64, then 128, then 256): it makes sense for it to grow, since the number of low-level features is often fairly low (e.g., small circles, horizontal lines), but there are many different ways to combine them into higher-level features. / Géron, Aurélien. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (Kindle Locations 10320-10322). O'Reilly Media. Kindle Edition.

The above says about the filter count of CNN with regard to its position(Lower or Higher).

As for the hidden layers, it used to be common to size them to form a pyramid, with fewer and fewer neurons at each layer — the rationale being that many low-level features can coalesce into far fewer high-level features. / Géron, Aurélien. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (Kindle Locations 6928-6930). O'Reilly Media. Kindle Edition.

And the above says about the neuron count of DNN with regard to is position(Lower or Higher).

It seems like the above two statements are in disagreement.

So what is correct? Lower-level features are more or higher-level feature are more?

$\endgroup$

1 Answer 1

2
$\begingroup$

They are not necessarily contradictory, since hidden representation in CNNs has also spatial dimension which decreases with depth.

In a CNN, the amount of information about the input sample in a hidden layer $n$ (let's call it $I(h_n)$) is size of the feature map times number of features: $$I(h_n) = \mathrm{width}\times\mathrm{height}\times{\#features}.$$ Here we mean width and height of the feature map. Usually, number of features is doubled whenever the resolution is halved, so $$I(h_{n+1})= \frac{\mathrm{width}} 2 \times\frac{\mathrm{height}} 2 \times{2\,\#features}$$ which is $\frac 1 2 I(h_n)$. Number of features increased, but the amount of information has halved, forcing the network to compress the information.

In this sense, both types of network behave similarily.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.