I've just started to learn about Neural Networks (more specificaly CNNs) and would like to clarify some points.
I've been using this tutorial for Neural Networks and this one for CNN.

Now I believe that I understand how Convolution, ReLU and Pooling are mathematically done, but I can't understand some other steps through the CNN process:

Suppose that we have 1 input image and 4 filters for the first convolution.

After the first convolution, how do we go from 4 feature maps to a bigger number of feature maps? I've seen examples where we go from 4 maps to 6 maps, which makes no sense to me. There is also this Link with a visual example, but I can't understand how to go from 6 Maps to 16 Maps at Convolution Layer 2 (this question was also asked HERE with more details but with no answer that I could understand)

  • 1
    $\begingroup$ One question for post, please. Two may be ok, but four definitely are not. Please split your questions in multiple posts, thanks. $\endgroup$ – DeltaIV May 10 '18 at 22:26
  • 1
    $\begingroup$ Ok, sorry about it... will split $\endgroup$ – GuiFGDeo May 10 '18 at 23:36
  • $\begingroup$ no problem :-) this is your first post and it's natural to be on a learning curve, but if you don't split the post, it risks being closed. As you can see, it has already been flagged for closure as too broad: even though I wasn't the one who flagged it, I think the flag is correct. $\endgroup$ – DeltaIV May 11 '18 at 9:18
  • $\begingroup$ Done, thanks and sorry... BTW, can I ask the other 3 points using the same "header" of this question? $\endgroup$ – GuiFGDeo May 11 '18 at 16:58
  • $\begingroup$ No: they are different questions, so each of them must have a different title, which gets "straight to the point" of the question. $\endgroup$ – DeltaIV May 11 '18 at 18:34

Each filter in a convolutional layer is required to have the same depth as the input volume, but you are free to choose the total number of filters used in the layer. Going from 4 maps to 6 maps would be accomplished using 6 filters, each of which was of depth 4. Each filter used in a convolutional layer corresponds to another "slice" (in the depth dimension) of the output volume of that layer.

  • $\begingroup$ But in this case not every filter would be applied to every map, beacuse otherwise we would have 6*4 final maps, each of depth 4, isnt that right? $\endgroup$ – GuiFGDeo May 14 '18 at 3:52

You should be familiar with fully connected neural networks, where the weights between a layer of $N$ input nodes and $M$ hidden nodes are stored in a $N$ by $M$ matrix.

With convolutional neural networks, the weights are stored in a $W$ by $H$ by $C$ by $D$ tensor (4d matrix) where $W$ is the width of the convolution window, $H$ is its height, and $C$ is its depth. The first 3 dimensions ($W$, $H$, $C$) are all input dimensions. So just imagine them as a very fancy $N$. The first hidden layer, (layer of filters), also has 3 dimensions, lets call them $W_2$, $H_2$ and $D$. $W_2$ and $H_2$ are calculated based off $W$ and $H$, so you cannot choose them, but you can choose $D$ which will be the depth of the first hidden layer of filters. So think of $W_2$, $H_2$ and $D$ as a fancy $M$.

In your example, to get from 4 feature maps to 6 feature maps, $C=4$ and $D=6$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.