1
$\begingroup$

This questions deals with the implementation of a convolution layer. First I like to make clear what I understand about cnn's.

  • A cnn uses filters/kernels to find geometrical features from the input image.
  • These filters have a pre-defined size, like 3x3x1 (for grayscale images).
  • Considering the mentioned filter, each neuron in the convolution layer has 9 (=3*3) weights, that will be learned. There are many neurons in each convolution layer, where every neuron has 9 weights to the input layer.

My conclusion is that, every neuron in the convolution layer is responsible for a small batch of the input data. E.g. the first filter can learn to detect a vertical line in the upper left corner. This trained filter may look like the following:

[[-1, 0, 1],
 [-1, 0, 1],
 [-1, 0, 1]]

Now my question: I know that each learned filter is applied to the full image. So the mentioned filter is able to detect vertical lines in every place in the input image (what I think is meant by weight sharing). In all tutorials I read, authors stated that the learned filter is slide over the image. But how is that achieved if there are only a small amount of connections (here 9) from every neuron to a specific area in the input data. In my understanding the input- and convolution-layer has to be fully connected to apply every filter to every subset of the image. But this is clearly not the case.

tl;dr: How are the filters in a cnn applied to the full image, if only a few weights per neuron in a convolution layer exists.

Any help would be appreciated.

Update: I think I found out what my misunderstanding is. Actually there are many connections between the input layer and each kernel in the convolution layer. But only the weights of each kernel, that is connected with the associated receptive field are learned. These learned weights are shared with all other connections, which are going from the input to this particular kernel. Imho this leads to the same effect like sliding each kernel over the input. Maybe someone can confirm?

I found the main idea in: LECUN, Yann, et al. Generalization and network design strategies. Connectionism in perspective, 1989, 19. Jg., Nr. 143-155, S. 18.

Improve this question
$\endgroup$
1
  • $\begingroup$ I like my drawing here. $\endgroup$
    – Dave
    Commented Apr 16, 2022 at 12:08

1 Answer 1

Reset to default
0
$\begingroup$

Not sure if I understand your question correctly.

But in general how filters are used in CNN is that they start a specific point on the image. For example the top left corner. At this location, we apply the filter n the 9 Pixels in that corner. After that we move the filter to the right by one row of pixels (often in practice its moved more than one row). Here again apply the filter (with the same weights) and move it another row to the right. I hope the gif can make that clear. https://gfycat.com/impartialdecimalbarnswallow

So we can use the same filter for the whole image.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your answer. So far I understand this concept. But I don't understand how the "move" part is implemented. In my opinion, there must be a weighted connection between the new input pixels (after we move the filter to the right) and our filter. Following this logic, after the filter is moved over the whole image, the filter is fully connected to every input pixel. Afaik this is not the case. $\endgroup$
    – Th0rn
    Commented Apr 11, 2022 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.