What is the difference between invariance to translation, covariance to translation and equivariance to translation? I get stuck at understanding the difference between invariance to translation, covariance to translation and equivariance to translation in the context of of convolutional neural network.
What does it mean :

*

*Convolution is equivariant to translation ?

*Convolution is convariant to translation ?

What is the difference between covariance and equivariance ?


*Are convolutional layers equivariant to translation ?

 A: There are two schools of thought when it comes to definitin of equivariance, covariance, invariance, and same-equivariance.

*

*Covariance is a concept often used in physics and is the same term as equivariance. Both are used when applying the transformation $\pi$ on the input of the function $f$ can be achieved by appying another transformation $\psi$ on the output of the function:
$f(\pi(x))=\psi(f(x))$


*Same-equivariance is an especial case of equivariance when $\psi=\pi$ (in some literature, same-equivariance is termed as equivariance, and instead, equivariance is termed covariance):
$f(\pi(x))=\pi(f(x))$


*Invariance is another especial case when the transformation $\psi$ is the identity function ($\psi=\mathbb{1}$)
$f(\pi(x))=f(x)$
Based on the above definitions:

*

*convolution is "equivariant" to translation

*convolution is also "same-equivariant" to translation,

*and since covariance is just another term for the same concept, convolution is "covariant" to translation.

Same is true for convolutional layers.
