In Alex Krizhevsky, et al. Imagenet classification with deep convolutional neural networks they enumerate the number of neurons in each layer (see diagram below).
The network’s input is 150,528-dimensional, and the number of neurons in the network’s remaining layers is given by 253,440–186,624–64,896–64,896–43,264– 4096–4096–1000.
A 3D View
The number of neurons for all layers after the first is clear. One simple way to calculate the neurons is to simply multiply the three dimensions of that layer (
planes X width X height):
- Layer 2:
27x27x128 * 2 = 186,624
- Layer 3:
13x13x192 * 2 = 64,896
However, looking at the first layer:
- Layer 1:
55x55x48 * 2 = 290400
Notice that this is not
253,440 as specified in the paper!
Calculate Output Size
The other way to calculate the output tensor of a convolution is:
If the input image is a 3D tensor
nInputPlane x height x width, the output image size will be
nOutputPlane x owidth x oheightwhere
owidth = (width - kW) / dW + 1
oheight = (height - kH) / dH + 1.
The input image is:
nInputPlane = 3
height = 224
width = 224
And the convolution layer is:
nOutputPlane = 96
kW = 11
kH = 11
dW = 4
dW = 4
(e.g. kernel size
Plugging in those numbers we get:
owidth = (224 - 11) / 4 + 1 = 54
oheight = (224 - 11) / 4 + 1 = 54
So we're one short of the
55x55 dimensions we need to match the paper. They might be padding (but the
cuda-convnet2 model explicitly sets the padding to 0)
If we take the
54-size dimensions we get
96x54x54 = 279,936 neurons - still too many.
So my question is this:
How do they get 253,440 neurons for the first convolutional layer? What am I missing?