The question is on the mathematical details of the convolutional neural networks. Assume that the architecture of the net (objective of which is image classification) is as such
- Input image 32x32
- First hidden layer 3x28x28 (formed by convolving with 3 filters of size 5x5, stride length = 0 and no padding), followed by activation
- Pooling layer (pooling over a 2x2 region) producing an output of 3x14x14
- Second hidden layer 6x10x10 (formed by convolving with 6 filters of size 5x5, stride length = 0 and no padding), followed by activation
- Pooling layer (pooling over a 2x2 region) producing an output of 6x5x5
- Fully connected layer (FCN) -1 with 100 neurons
- Fully connected layer (FCN) -2 with 10 neurons
From my readings thus far, I have understood that each of the 6x5x5 matrices are connected to the FCN-1. I have two questions, both of which are related to the way output from one layer is fed to another.
- The output of the second pooling layer is 6x5x5. How are these fed to the FCN-1? What I mean is that each neuron in the FCN-1 can be seen as node that takes a scalar as input (or a 1x1 matrix). So how do we feed it an input of 6x5x5? I initially thought we’d flatten out the 6x5x5 matrices and convert it into a 150x1 array and then feed it to the neuron as if we have 150 training points. But doesn’t flattening out the feature map defeat the argument of spatial architecture of images?
- From the first pooling layer we get 3 feature maps of size 14x14. How are the feature maps in the second layer generated? Lets say I look at the same region (a 5x5 area starting from the top left of the feature maps) across the 3 feature maps I get from the first convolutional layer. Are these three 5x5 patches used as separate training examples to produce the corresponding region in the next set of feature maps? If so then what if the three feature maps are instead RGB values of an input image? Would we still use them as separate training examples?