I'm having trouble understanding how the math works in the C5 convolution layer of the LeNet-5 network.
Layer 4 (S4) has 16 feature maps of size 5x5. Using a convolution kernel of size 5x5 with valid padding on this layer would produce a 1x1 output. Therefore, if a 5x5 kernel were used with a single filter per kernel, C5 would produce 16 1x1 feature maps.
The network architecture actually shows 120 1x1 feature maps. I would assume that this would be achieved by using multiple filters per kernel. However, in order to get 120 feature maps from 16 you'd need 7.5 filters per kernel which doesn't really make sense (16 * 7.5 = 120).
I'm assuming I'm missing something simple here, but how does the math work out to go from 16 5x5 feature maps to 120 1x1 feature maps? How do you achieve this outcome if not by some integer value of filters per kernel? Thanks.