# How are kernels applied to feature maps to produce other feature maps?

I am trying to understand the convolution part of convolutional neural networks. Looking at the following figure:

I have no problems understanding the first convolution layer where we have 4 different kernels (of size $k \times k$), which we convolve with the input image to obtain 4 feature maps.

What I do not understand is the next convolution layer, where we go from 4 feature maps to 6 feature maps. I assume we have 6 kernels in this layer (consequently giving 6 output feature maps), but how do these kernels work on the 4 feature maps shown in C1? Are the kernels 3-dimensional, or are they 2-dimensional and replicated across the 4 input feature maps?

• I am stuck in the same place. Unfortuantely Yann Lecun-s paper does not explain that too - I have been going through several pdfs and videos of the last few days and everyone seems to skip that part. Yann Lecun's paper actually talks of 6 to 16 feature maps with a mapping table in layer 2. First output feature map gets input from 0,1,2 input feature maps. But that output feature map is 10 by 10, the 3 input feature maps being 14 by 14. So how did that work ? Did you understand whats going on ? Is it a 3-D kernel ? or is it averaging the outputs from the location*kernel (convolution)? – Run2 Nov 13 '14 at 7:28

The kernels are 3-dimensional, where width and height can be chosen, while the depth is equal to the number of maps in the input layer - in general.

They are certainly not 2-dimensional and replicated across the input feature maps at the same 2D location! That would mean a kernel wouldn't be able to distinguish between its input features at a given location, since it would use one and the same weight across the input feature maps!

There is not a one-to-one correspondence between layers and kernels necessarily. That depends on the particular architecture. The figure you posted suggests that in the S2 layers you have 6 feature maps, each combining all feature maps of the previous layers, i.e. different possible combinations of the features.

Without more references I cannot say much more. See for example this paper

• I am looking at LeNet-5 in particular, and using this deeplearning.net/tutorial/lenet.html as my reference. It seems from that page, that the kernels are 3-dimensional, but it is not 100% clear to me. – utdiscant Feb 7 '14 at 15:50
• You need to read this paper then (yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf). On page 8 it is described how the different layers are connected. As I said, layer each feature at layer combines several features from previous layer at the same location. – jpmuc Feb 7 '14 at 16:59
• The link is dead. – jul Sep 17 '15 at 7:52

Table 1 and Section 2a of Yann LeCun's "Gradient Based Learning Applied to Document Recognition" explains this well: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf Not all regions of the 5x5 convolution are used to generate the 2nd convolutional layer.