# How does Krizhevsky's '12 CNN get 253,440 neurons in the first layer?

In Alex Krizhevsky, et al. Imagenet classification with deep convolutional neural networks they enumerate the number of neurons in each layer (see diagram below).

The network’s input is 150,528-dimensional, and the number of neurons in the network’s remaining layers is given by 253,440–186,624–64,896–64,896–43,264– 4096–4096–1000.

### A 3D View

The number of neurons for all layers after the first is clear. One simple way to calculate the neurons is to simply multiply the three dimensions of that layer (planes X width X height):

• Layer 2: 27x27x128 * 2 = 186,624
• Layer 3: 13x13x192 * 2 = 64,896
• etc.

However, looking at the first layer:

• Layer 1: 55x55x48 * 2 = 290400

Notice that this is not 253,440 as specified in the paper!

### Calculate Output Size

The other way to calculate the output tensor of a convolution is:

If the input image is a 3D tensor nInputPlane x height x width, the output image size will be nOutputPlane x owidth x oheight where

owidth = (width - kW) / dW + 1

oheight = (height - kH) / dH + 1 .

The input image is:

• nInputPlane = 3
• height = 224
• width = 224

And the convolution layer is:

• nOutputPlane = 96
• kW = 11
• kH = 11
• dW = 4
• dW = 4

(e.g. kernel size 11, stride 4)

Plugging in those numbers we get:

 owidth = (224 - 11) / 4 + 1 = 54 oheight = (224 - 11) / 4 + 1 = 54 

So we're one short of the 55x55 dimensions we need to match the paper. They might be padding (but the cuda-convnet2 model explicitly sets the padding to 0)

If we take the 54-size dimensions we get 96x54x54 = 279,936 neurons - still too many.

So my question is this:

How do they get 253,440 neurons for the first convolutional layer? What am I missing?

• Have you ever solved this? Just to be pedantic with your calculations: owidth and oheight would actually be 54.25. I tried to figure this out, and as a first step divided the supposed 253440 neurons among 96 filters, which yields 2640 neurons per filter. This isn't a square number. So either we both have a misunderstanding here, or there might be a mistake by the authors... Have you contacted them? – anderas Feb 21 '15 at 22:25
• same with me, this is very confuse me. btw there is true the input is 224x224x3? i think it must be 227x227x3. let we see if we have 227x227, 5 cell on first left and 5 cell on last right cannot be the center of kernel convolution with size 11x11. So the first center of kernel is cell (6,6) and the last of center kernel in first row is cell(6x222). With stride-4 we will get the center of kernel on row-sixth are: cell on column :6,10,14, ...,222 and simple formulation for the center of kernel-k is on column = 6+(k-1)*4 so that column 222 is the k-th center = (222-6)/4 +1 = 55. – user89674 Sep 17 '15 at 7:09
• Note that 48*48*55*2=253440, so it's possible they had a typo when calculating the number of neurons in the first layer (multiplied by 48 instead of 55). – tsiki Nov 19 '15 at 17:52
• – kenorb Jun 27 '16 at 18:55
• @Firebug This is an interesting usage of the [references] tag. I thought we use it only for questions that ask for references. But perhaps I was wrong. Do you use it differently? – amoeba Sep 23 '16 at 19:50