1
$\begingroup$

I am learning CNN with TensorFlow and Python.

I do not understand the connection between layer $\ell$ and layer $\ell+1$. For example, for the input image and the first layer, it is easy as there is only one input, and hence there is as many feature maps as filters, and each filter gets to be 'multiplied' by the input image. The resulting feature map is of size ((input_size - filter_size + 2*padding) / stride) + 1.

A similar question and answer clearly responds to his particular example: How are filters and activation maps connected in Convolutional Neural Networks?

But in general I still don't understand. When we build a CNN, for example:

model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape)) 
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))

How does TensorFlow make the connection between 32 features map to the next feature map? Typically I would think that with 64 filters and 32 feature maps from the previous layer we would get 64*32 feature maps in the next layer (all features are connected to each filter). But I think that above code will result in 64 feature maps.

|cite|improve this question
$\endgroup$
3
$\begingroup$

Your confusion stems from the fact that channels (feature maps) are treated somewhat differently than other dimensions.

Let's say you have a grayscale image input to the first layer and 32 kernels of shape (3,3) as per your example. But in fact, those kernels have shape (3,3,1) - 1 for the number of channels in the input. For a RGB input image it would be 3. The number of channels is simply omitted in the code because it is inferred automatically from the number of channels of the layer input.

The output of this layer has 32 channels (1 per each kernel). In the second layer in your example, you have 64 kernels of shape (3,3), but they are in fact (3,3,32)! Each of these kernels is aggregating information from all input feature maps.

Typically I would think that with 64 filters and 32 feature maps from the previous layer we would get 64*32 feature maps in the next layer (all features are connected to each filter).

I hope that from the above explanation it is clear that you are not applying each of the 64 kernels on each of the 32 feature maps individually. Instead, each of these 64 kernels is looking at all of the 32 feature maps at the same time, having different weights for each of them.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you so much for your nice explanation and quick answer @jan kukacka! Did I understood well that actually each layer output can be seen as one image with a certain channel size? in this case the 'input-image' for the layer having 64 kernels, have 32 channels (or 32*3 if the initial image was RGB)? $\endgroup$ – miki Sep 4 '18 at 11:33
  • $\begingroup$ The input image for the second layer will always be an "image" with 32 channels, regardless if the original input was RGB or grayscale. $\endgroup$ – Jan Kukacka Sep 4 '18 at 11:34
  • $\begingroup$ Ah yes true! Its because each conv is resulting in a feature map with channel of size 1 in my understanding, so if the initial input image is RGB, then the layer with 32 kernel will induce 32 conv with a (3,3,3)kernel and image(N,N,3) each of these 32 resulting a feature map of size (N-3+1,N-3+1,1) (stride=1,Padding=0). Hence the 'input-image' for the layer having 64 kernels has size (N-3+1,N-3+1,32) and not (N-3+1,N-3+1,32*3) . Is it correct? Thank you for your big help I was (or am) missing a big thing!! $\endgroup$ – miki Sep 4 '18 at 11:50
  • $\begingroup$ Yes, that's correct. Glad to help! Don't forget to mark the answer and upvote if it indeed solved your problem :) $\endgroup$ – Jan Kukacka Sep 4 '18 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.