0
$\begingroup$

I understand the concept behind why convolution layers / max pool operations work, but I cannot conceptualize how they are applied in typical neural network model.

For example if I had a NN model that looked like this:

[inputs] -(weights 0)-> [conv. layer / max-pooling] -(weights 1)-> [dense layer] -(weights 2)-> [output]

where exactly would the operations for apply filters/averaging take place? During the weights section or the [conv. layer / max-pooling] section itself

Assuming the inputs take in a 28 * 28 pixel, how would I take a 2 * 2 filter and change the original matrix, would I have to split the inputs into 2 * 2 chunks and recombine them?

If I wanted to preform max-pooling on the same inputs would I also have to also split the image into 2x2 chunks to downscale the image and add all of those little chunks together to form a new matrix?

[2][3] [5][4] ---> [5]

I'm currently at the phase of transitioning from theory to practice.

| cite | improve this question | |
$\endgroup$
2
$\begingroup$

where exactly would the operations for apply filters/averaging take place? During the weights section or the [conv. layer / max-pooling] section itself

Weights actually belong to filters, i.e. the convolutional layer. Pooling occur afterwards and doesn't relate to the weights.

Assuming the inputs take in a 28 * 28 pixel, how would I take a 2 * 2 filter and change the original matrix, would I have to split the inputs into 2 * 2 chunks and recombine them?

You convolve the input image and the filter. You can convolve different sized images (signals), no need to split the larger one into smaller chunks. This step sometimes can slightly differ from the usual convolution, in which zero-padding can be used, or reversal of filter (which isn't important in learning) is not performed etc.

If I wanted to perform max-pooling on the same inputs would I also have to also split the image into 2x2 chunks to downscale the image and add all of those little chunks together to form a new matrix?

After the conv. layer, you divide the image into 2x2 non-overlapping chunks, choose the max amongst the 4 pixels, and form a new image. Yes, this is a form of downscaling, but not in the usual sense in signal processing, where you apply, for example, low-pass filters and subsample.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Ah that explains a lot, so just to clarify, where the weights normally are exists a function that's job it is to apply a filter for convolutions, while the max pooling takes place within the max-pooling layer itself? At this max pooling layer, are weights and activation functions also applied or do the layers only preform max-pooling and convolutions without preforming any of the dense layer functions. $\endgroup$ – Jatearoon Keene Boondicharern Feb 1 '19 at 18:21
  • $\begingroup$ Pooling layer doesn't contain weights/activations, just pooling (mean, max or whatever you choose) Conv. layer has filters (i.e. weights) and sometimes activations. $\endgroup$ – gunes Feb 1 '19 at 18:44
  • $\begingroup$ Oh sorry, I tend to forget to do that $\endgroup$ – Jatearoon Keene Boondicharern Feb 2 '19 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.