How does convolution work?

Question

As far I understood, a CNN is trying to represent an image's characteristics (WxHxRGB) in a 1 dimensional matrix. I really like the idea, because it makes a lot of sense in images to represent the relations of the pixels vs flattening it into a 1D matrix, and I'm pretty sure it can be useful in cases other than images as well.

Pooling, and activation layers are clear, however, convolution, and fully connected layers were clear, but as I read more about it, they became confusing again.

At the moment in my mind convolution is applying K number of filters with size F, adding P amount of zero paddings as "border" (defaults to 0), with S as spatial size (defaults to 1), which means... what does it actually mean? I saw it explained as the filter will jump S "pixels" on the input once the current area is calculated. Would make sense. See this discussion regarding to what happens if the input size equals to the filter size, and setting a spatial size to basically anything. As logically if the sizes matching, it wouldn't have a chance to "jump". See the discussion.

With regarding to filters, in my understanding the filters depth must match the input depth, and the output depth will match K, the number of filters, and each element in the filter are weights, and optionally each filter can have a bias. After the forward pass, in the backpropagation they will be adjusted (because of the pooling, relu, etc., can't understand how it can work).

There's a definition parameter sharing. Sharing what parameters, where, in which case? Sharing the filter size (F), or spatial size (S), and sharing it for just one convolutional layer, or all of them?

There's another discussion which brings up some questions: here, which is about this.

With regards to converting FC (MLP) layers to convolutional layers, it says:

• Replace the first FC layer that looks at [7x7x512] volume with a CONV layer that uses filter size F=7, giving output volume [1x1x4096].
• Replace the second FC layer with a CONV layer that uses filter size F=1, giving output volume [1x1x4096]
• Replace the last FC layer similarly, with F=1, giving final output [1x1x1000]

What are those magical numbers like 4096, and 1000?

Also, how they are calculated? It does specify the F value, which explains the width, and height, but it doesn't specify K. And in my mind the only way the depth can be changed is specifying K as the depth we want. So to have 1x1x4096, from 7x7x512, is with F 7, K 4096. And to get 1x1x1000 from 1x1x4096 is to set F to 1, and K to 1000. But why? Why not just get the output 1x1x1000 immediately from 7x7x512? Also are those numbers just random, or is there a specific reasoning behind them?

Regarding to converting fully connected layers to convolutional layers, I've read about completely replacing FCs, and getting a fully convolutional network. Does it mean that for example if I have a convolutional or pooling layer at the end which resulting in 1x1x3, it can be suitable for classification, like the animal on a picture is a dog/cat/horse?

Answer 1

At the moment in my mind convolution is applying K number of filters with size F, adding P amount of zero paddings as "border" (defaults to 0), with S as spatial size (defaults to 1), which means... what does it actually mean?

The common names for those terms are (K) number of channels, (F) filter size, (P) padding, (S) stride. Finally, there's also "dilation". You can find pretty good illustrations of what exactly they do, for example here.

With regarding to filters, in my understanding the filters depth must match the input depth, and the output depth will match K

The number of input channels is completely independent from the number of output channels. If you understand convolution for 1 input and 1 output channel, to understand convolution with N inputs and M outputs, just imagine you do N*M convolutions, then sum over the N inputs so that you're left with M channels.

There's a definition parameter sharing. Sharing what parameters, where, in which case? Sharing the filter size (F), or spatial size (S), and sharing it for just one convolutional layer, or all of them?

Parameter sharing in the context of convolutions just refers to the fact that if you were to rewrite a convolution as a single matrix multiplication (which you can, since it's a linear transform), many of the entries (parameters) in the matrix would be repeated, or "shared".

What are those magical numbers like 4096, and 1000?

They're references to one particular well-known convolutional network, and it just happens that the penultimate layer of that network has 4096 channels, and the last layer has 1000 channels.

Does it mean that for example if I have a convolutional or pooling layer at the end which resulting in 1x1x3, it can be suitable for classification, like the animal on a picture is a dog/cat/horse?

Yep.