2
$\begingroup$

CNN (convolutional neural networks) are well-known to have the nice property of "translation invariance".

Is there any other type of neural network that does not have such a property?

Or can we remove certain "layers" in CNN (such as max pooling, dropout, etc.) to "disable" translation invariance?

Possible scenarios is to:

  • classify by position (e.g. cat in the top left vs cat in the bottom right of image)

  • classified by "skewness" of object (e.g. classify squares vs rectangles).

Thanks a lot for any help.

Improve this question
$\endgroup$
2
  • $\begingroup$ The premise of the question is false. Invariance is introduced not by convolution, but by max-pooling. $\endgroup$
    – Frans Rodenburg
    Oct 31, 2019 at 15:22
  • $\begingroup$ To add to the previous comment: translation invariance in CNNs with pooling layers is limited to the spatial resolution of the final convolutional layer. If your final conv layer has dimensions [5 x 5 x 128], where the first two are the spatial dimensions and the last value is the number of feature maps, then a subsequent fully-connected layer can learn to weight different spatial locations differently in a way that is not translation invariant. So unless the spatial dimensions are fully "pooled-out" before introducing FC layers, there may be some residual translation dependence. $\endgroup$
    – Ruben van Bergen
    Oct 31, 2019 at 17:49

1 Answer 1

Reset to default
3
$\begingroup$

While CNNs are not completely translation invariant, common architectures are... not ideal for completing tasks where precise "positional information" is important -- for example, your example of finding the position of a cat.

CoordConv solves this with a dead simple solution: add two more channels to the input specifying the coordinate of each input pixel.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.