The convolution is an operation on two functions of a real- valued argument and is typically denoted with an asterisk:

s(t) = (x ∗ w)(t)

It is a special kind of linear operation which is also commutative, so:

s(t) = (x ∗ w)(t) is also equal to s(t) = (w ∗ x)(t).

In convolutional neural networks we usually use convolutions over multiple axis. In a two-dimensional image we want to use 2 convolutional operations over two axis at a time. We also use a 2-D kernel as follows:

enter image description here

where I is the input, and K is the Kernel.

It's commutative equivalent is:

enter image description here

As per www.deeplearningbook.org,

Usually the latter formula is more straightforward to implement in a machine learning library, because there is less variation in the range of valid values of m and n.

I am struggling to comprehend this sentence and thus, my question is, why would there be less variation in the range of valid values of m and n when they are commutative equivalents?

  • $\begingroup$ The domain of the kernel $K$ is usually smaller--often much smaller--than the domain of the input over which it is convolved. If you were to be rigorous in your summation notation you would explicitly indicate the range of subscripts and this fact would become obvious. $\endgroup$ – whuber Oct 31 '18 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.