# How are kernels applied to feature maps to produce other feature maps?

I am trying to understand the convolution part of convolutional neural networks. Looking at the following figure:

I have no problems understanding the first convolution layer where we have 4 different kernels (of size $k \times k$), which we convolve with the input image to obtain 4 feature maps.

What I do not understand is the next convolution layer, where we go from 4 feature maps to 6 feature maps. I assume we have 6 kernels in this layer (consequently giving 6 output feature maps), but how do these kernels work on the 4 feature maps shown in C1? Are the kernels 3-dimensional, or are they 2-dimensional and replicated across the 4 input feature maps?

• I am stuck in the same place. Unfortuantely Yann Lecun-s paper does not explain that too - I have been going through several pdfs and videos of the last few days and everyone seems to skip that part. Yann Lecun's paper actually talks of 6 to 16 feature maps with a mapping table in layer 2. First output feature map gets input from 0,1,2 input feature maps. But that output feature map is 10 by 10, the 3 input feature maps being 14 by 14. So how did that work ? Did you understand whats going on ? Is it a 3-D kernel ? or is it averaging the outputs from the location*kernel (convolution)? – Run2 Nov 13 '14 at 7:28

The kernels are 3-dimensional, where width and height can be chosen, while the depth is equal to the number of maps in the input layer - in general.

They are certainly not 2-dimensional and replicated across the input feature maps at the same 2D location! That would mean a kernel wouldn't be able to distinguish between its input features at a given location, since it would use one and the same weight across the input feature maps!

There is not a one-to-one correspondence between layers and kernels necessarily. That depends on the particular architecture. The figure you posted suggests that in the S2 layers you have 6 feature maps, each combining all feature maps of the previous layers, i.e. different possible combinations of the features.

Without more references I cannot say much more. See for example this paper

• I am looking at LeNet-5 in particular, and using this deeplearning.net/tutorial/lenet.html as my reference. It seems from that page, that the kernels are 3-dimensional, but it is not 100% clear to me. – utdiscant Feb 7 '14 at 15:50
• You need to read this paper then (yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf). On page 8 it is described how the different layers are connected. As I said, layer each feature at layer combines several features from previous layer at the same location. – jpmuc Feb 7 '14 at 16:59
• The link is dead. – jul Sep 17 '15 at 7:52

Table 1 and Section 2a of Yann LeCun's "Gradient Based Learning Applied to Document Recognition" explains this well: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf Not all regions of the 5x5 convolution are used to generate the 2nd convolutional layer.

It doesn't really answers the question because it explains only the first convolution layer, but contains good explanation of basic intuition about convolution in CNNs. It also describes deeper mathematical definition of convolution. I think it is related to question topic.

• Welcome to the site. We are trying to build a permanent repository of high-quality statistical information in the form of questions & answers. Thus, we're wary of link-only answers, due to linkrot. Can you post a full citation & a summary of the information at the link, in case it goes dead? – gung - Reinstate Monica Mar 31 '15 at 13:12
• @gung, thank you for the notice, sorry for misunderstanding the concepts. The situation is: this article doesn't really answers the question, but when I was looking for basic intuition about CNNs I found this question and I hoped to help with this article to someone who also search for basic intuition and got this question. Ok, better to delete it, yes? Thank you. – Anatoly Vasilyev Mar 31 '15 at 15:22
• I think it would be fine to say, 'this article may serve as food for thought, but doesn't completely answer the question', or something like that. There may well be value here. Just give a complete citation, & give a summary of the information contained, in case the link goes dead. – gung - Reinstate Monica Mar 31 '15 at 15:44
• Thanks for the extra information. Can you provide a complete citation for the paper (author, year, title, journal, etc) & a summary of its content? – gung - Reinstate Monica Mar 31 '15 at 16:15
• @gung yes, of course. But seems like this article is only in this blog, so I couldn't find any other useful information about it. Thank you for clarifying my view – Anatoly Vasilyev Mar 31 '15 at 16:25