Recently I have started to implement my own Convolutional Neural Network. I have few questions. I will talk with reference to an example, so that we all remain on the same page. Suppose,
64X64X1 that is gray-channel only.------------Output - 64X64X1
5X5X6 that is 6
conv_maps, each of size 5X5-Output - 60X60X6
P1: Max-Pooling - non_overlapping
size = 2X2--Output - 30X30X6
9X9X8 - 8
conv_maps, each of size 9X9--------Output - 22X22X48
P2: Max-Pooling - Non_overlapping
size = 2X2--Output - 11X11X48
Ok, Now following are the questions:
As I understand, ReLU is applied to every neuron. That is, in C1, first time
5X5patch is moved over
input- Then the sum of convolution has to pass through
transform_function. And no
transform_functionat Pooling layer. Am I correct in understanding it?
Which function to use as
transfer_function?Softplus? Noisy one? Leaky one?
Also, same transfer function should be used for
FeedForwardpart, right? Or can I change to
- Convolution-Feature_Map Connections
How to carry out next convolution? The
P1layer has 6 maps of
30X30. There are going to be 8 convolutional kernels, each of size 9X9. But I have NEVER seen this producing
LeNethas output of 16 maps. How to produce those maps is given in this paper on page 8. After reading it again and again I DO NOT get how to generate next feature maps. Are they doing it like this -->
- Also, isn't the method mentioned in the paper specific to 'OCR'? I am very confused about how to write program for them in a user-friendly way. For e.g. if I want to see the output of different architecture, how to define these rules of connections programmatically?
I definitely did not understand "It forces a break of symmetry .." thing from the above mentioned paper. Please if you could elaborate. I am not able to visualize problem of symmetry here.
- About Bias
Initially I thought
biasas a window of kernel size, but now I think its just a number between 0-1. But How do I add a bias? If I treat kernel as a matrix, say 5X5, then how possibly I can add a single number to matrix? We get the sum after the convolution, I think I am supposed to add the bias to this sum and then apply the transform function. Right?