Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In the deep learning book by Goodfellow et al., it is stated
"Convolution is not naturally equivariant to some other transformations, such as changes in the scale or rotation of an image. Other mechanisms are necessary for handling these kinds of transformations."
What are the mechanims commonly used to deal with differences in scale?
What is currently state of the art?
One commonly employed method in object detection (this task is particularly sensitive to variations in scale!) is feature pyramids -- essentially the neural network produces a "pyramid" of multi-scale feature maps, with the highest resolution maps at the bottom of the pyramid and the lowest resolution at the top. The high resolution feature maps are useful for detecting small objects, and the low resolution maps for large objects.
Another more general approach is spatial transformers -- a module which can apply arbitrary affine transformations in order to rotate and scale the image however is ideal. However this comes at the cost of additional computation expense.
