Help to fully understand Convolutional Neural Networks

Question

I've just started to learn about Neural Networks (more specificaly CNNs) and would like to clarify some points.
I've been using this tutorial for Neural Networks and this one for CNN.

Now I believe that I understand how Convolution, ReLU and Pooling are mathematically done, but I can't understand some other steps through the CNN process:

Suppose that we have 1 input image and 4 filters for the first convolution.

After the first convolution, how do we go from 4 feature maps to a bigger number of feature maps? I've seen examples where we go from 4 maps to 6 maps, which makes no sense to me. There is also this Link with a visual example, but I can't understand how to go from 6 Maps to 16 Maps at Convolution Layer 2 (this question was also asked HERE with more details but with no answer that I could understand)

Answer 1

Each filter in a convolutional layer is required to have the same depth as the input volume, but you are free to choose the total number of filters used in the layer. Going from 4 maps to 6 maps would be accomplished using 6 filters, each of which was of depth 4. Each filter used in a convolutional layer corresponds to another "slice" (in the depth dimension) of the output volume of that layer.

Answer 2

You should be familiar with fully connected neural networks, where the weights between a layer of $N$ input nodes and $M$ hidden nodes are stored in a $N$ by $M$ matrix.

With convolutional neural networks, the weights are stored in a $W$ by $H$ by $C$ by $D$ tensor (4d matrix) where $W$ is the width of the convolution window, $H$ is its height, and $C$ is its depth. The first 3 dimensions ($W$, $H$, $C$) are all input dimensions. So just imagine them as a very fancy $N$. The first hidden layer, (layer of filters), also has 3 dimensions, lets call them $W_2$, $H_2$ and $D$. $W_2$ and $H_2$ are calculated based off $W$ and $H$, so you cannot choose them, but you can choose $D$ which will be the depth of the first hidden layer of filters. So think of $W_2$, $H_2$ and $D$ as a fancy $M$.

In your example, to get from 4 feature maps to 6 feature maps, $C=4$ and $D=6$.