3
$\begingroup$

image

In C3 layer of Convolutional Neural Network (LeNet 5) we need to construct 16 feature maps of size of 10*10. In C3 layer a connection map is used (this connection map shows what feature maps from layer S2 connected with S3 feature maps), for example feature map number 9 in C3 is connected with feature maps 3,4,5,6.

But for example if we need to construct first feature map of C3, we will use 1,2,3 feature maps of S2 according to table. Also I have read that "As shown in table 1 the choice is made not to connect every feature map of S2 to every feature map of C3. Each unit of C3 is connected to several receptive fields at identical locations in a subset of feature maps of S2". According to this statement unit is a neuron and it is connected to several receptive fields of C3 feature maps.

But as we building first feature maps of C3, we will use 1,2,3 feature maps of S2 layer. But in total it will give 3 feature maps, it is wrong. In total we have 100 neurons and 100 receptive fields in order to built one 10*10 feature maps of C3. Even if we will use 3 receptive fields as stated in table (from feature maps 1,2,3 of S2 layer).

In total we will have 3 receptive fields of 5*5 size, finally it will give us a 3 neurons and 3 pixels which is ain't enough to build one 10*10 feature map. Could you please say how exactly should a C3 layer feature maps must be constructed? Thank you!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Your input maps from S2 are of size 14x14.

To build your first feature map of C3, you convolve 3 of your input maps with 5x5 filters, which gives you 3 10x10 maps that are summed up to give your first feature map, which is then of size 10x10.

More specifically, if we call $S^k$ the k-th input map, $C^l$ the l-th feature map, and $W^{k,l}$ the convolution filter that convolve map $k$ into $l$, then we got :

$C^1 = S^1 *W^{1,1} + S^2 *W^{2,1} + S^3 *W^{3,1}$

with $*$ denoting 2-d convolution.

Also, a non-linearity and a pooling operation are usually applied to each $C^k$ before computing the next layer.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for explanation! As we need to build 16 feature maps and use 5*5 kernel (filter). During constructing first feature map of C3 we will use distinct 3 5*5 kernels or just one same kernel. And in other 15 remaining feature maps we will use this kernel which we have used during constructing first feature map? Or they will be also various? Can we generate and fill this kernels with random small numbers as we did it during creating 6 kernels on C1? $\endgroup$
    – Asus_ROG
    Commented Aug 10, 2015 at 11:32
  • $\begingroup$ Also you have explained that "Also, a non-linearity and a pooling operation are usually applied to each Ck before computing the next layer."Could you please say what is described under non-linearity and pooling operation. If we will produce pooling operation feature map will reduce in size and will be less than 10*10 or it is another pooling operation not that which is used on S2,S4 layers. And non-linearity, pooling applied to Cl after we had computed it by the formula below? You have said that this operation is "applied to each Ck before computing the next layer", maybe to next feature map? $\endgroup$
    – Asus_ROG
    Commented Aug 10, 2015 at 11:37
  • $\begingroup$ Kernels $W^{k,l}$ are dependant of k and l, meaning they are all distinct kernels that capture different kinds of information. You can initialize them with random values. For example in this tutorial, they pick random numbers between $[-d,+d]$ where $d=\sqrt{\frac{6}{numberOfConnectedNeurons}}$ $\endgroup$
    – Cloud Skywalker
    Commented Aug 10, 2015 at 12:53
  • $\begingroup$ Non-linearity are functions that will answer a specific point about the score of a neuron. Usually, $sigmoid$ or $tanh$ are picked because they will alleviate the weight of big scores for the next operation. Pooling operation will reduce the resolution of the map as you said. It is probably the same operation that you are talking about on S2 and S4 layers. Note that hyper-parameters and construction of a lenet model are not fixed and depend on the dataset. $\endgroup$
    – Cloud Skywalker
    Commented Aug 10, 2015 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.