I have a couple of questions that are confusing me regarding the CNN.
1) The features extracted using CNN are scale and rotation invariant?
2) The Kernels we use to convolution with our data are already defined in the literature? what kind of these kernels are? is it different for every application?
4 Answers
1) The features extracted using CNN are scale and rotation invariant?
A feature in itself in a CNN is not scale or rotation invariant. For more details, see: Deep Learning. Ian Goodfellow and Yoshua Bengio and Aaron Courville. 2016: http://egrcc.github.io/docs/dl/deeplearningbook-convnets.pdf ; http://www.deeplearningbook.org/contents/convnets.html:
Convolution is not naturally equivariant to some other transformations, such as changes in the scale or rotation of an image. Other mechanisms are necessary for handling these kinds of transformations.
It is the max pooling layer that introduces such invariants:
2) The Kernels we use to convolution with our data are already defined in the literature? what kind of these kernels are? is it different for every application?
The kernels are learnt during the training phase of the ANN.
-
-
1$\begingroup$ @Franck 1) That means, we don't take any special steps to make over system rotation invariant? and how about the scale invariant, is it possible to get the scale invariant from the max pooling? $\endgroup$ Oct 8, 2016 at 2:08
-
8$\begingroup$ Note that the pooling you're talking about is refered to as cross-channel pooling and is not the type of pooling which is usually refered to when talking about "max-pooling", which only pools over spatial dimensions (not over different input channels). $\endgroup$– SoltiusJun 13, 2018 at 15:35
-
2$\begingroup$ Does this imply a model which does not have any max-pool layers (most of the current SOTA architectures do not use pooling) is completely scale dependent? $\endgroup$ Aug 10, 2018 at 7:46
-
1$\begingroup$ So does pooling make the network only slightly invariant to scaling and rotation? $\endgroup$– ShashwatMay 1, 2020 at 18:33
I think there are a couple of things confusing you, so first things first.
Given a signal $x[n]$, and a kernel (also called a filter) $h[n]$, then the convolution of $x[n]$ with $h[n]$ is written as $y[n] = (x \star h)[n]$, and is computed via a sliding dot-product, mathematically given by:
$$ y[n] = \sum_{m=-\infty}^{\infty}x[m] \ h[n-m] $$
The above if for one-dimensional signals, but the same can be said for images, which are just two-dimensional signals. In that case, the equation becomes:
$$ I_{new}[r,c] = \sum_{u=-\infty}^{\infty}\sum_{v=-\infty}^{\infty} I_{old}[u,v] \ k[r-u, c-v] $$
Pictorially, this is what is happening:
At any rate, the thing to keep in mind, is that the kernel, in actually learned during training a Deep Neural Network (DNN). A kernel is just going to be what you convolve your input with. The DNN will learn the kernel, such that it brings out certain facets of the image (or previous image), that are going to be good for lowering the loss of your target objective.
This is the first crucial point to understand: Traditionally people have designed kernels, but in Deep Learning, we let the network decide what the best kernel should be. The one thing we do specify however, is the kernel dimensions. (This is called a hyperparameter, for example, 5x5, or 3x3, etc).
-
2$\begingroup$ Nice explanation. Can you please answer the First part of the question. About the CNN is scale/rotation Invariant? $\endgroup$ Oct 11, 2016 at 4:58
-
1
Many authors including Geoffrey Hinton (who proposes Capsule net) try to solve the issue, but qualitatively. We try to address this issue quantitatively. By having all convolution kernels be symmetric (dihedral symmetry of order 8 [Dih4] or 90-degree increment rotation symmetric, et al) in the CNN, we would provide a platform for the input vector and resultant vector on each convolution hidden layer be rotated synchronously with the same symmetric property (i.e., Dih4 or 90-increment rotation symmetric, et al). Additionally, by having the same symmetric property for each filter (i.e., fully connected but weighs sharing with the same symmetric pattern) on the first flatten layer, the resultant value on each node would be quantitatively identical and lead to the CNN output vector the same as well. I called it transformation-identical CNN (or TI-CNN-1). There are other methods that can also construct transformation-identical CNN using symmetric input or operations inside the CNN (TI-CNN-2). Based on the TI-CNN, a geared rotation-identical CNNs (GRI-CNN) can be constructed by multiple TI-CNNs with the input vector rotated by a small step angle. Furthermore, a composed quantitatively identical CNN can also be constructed by combining multiple GRI-CNNs with various transformed input vectors.
"Transformationally Identical and Invariant Convolutional Neural Networks through Symmetric Element Operators” https://arxiv.org/abs/1806.03636 (June 2018)
“Transformationally Identical and Invariant Convolutional Neural Networks by Combining Symmetric Operations or Input Vectors” https://arxiv.org/abs/1807.11156 (July 2018)
"Geared Rotationally Identical and Invariant Convolutional Neural Network Systems" https://arxiv.org/abs/1808.01280 (August 2018)
I think max pooling can reserve translational and rotational invariances only for translations and rotations smaller than the stride size. If greater, no invariance
-
1$\begingroup$ could you expand a little bit? We encourage answers on this site to be a little bit more detailed than this (right now, this looks more to a comment). Thank you! $\endgroup$– AntoineMar 17, 2019 at 8:12