# What is the difference between invariance to translation, covariance to translation and equivariance to translation?

I get stuck at understanding the difference between invariance to translation, covariance to translation and equivariance to translation in the context of of convolutional neural network.

What does it mean :

1. Convolution is equivariant to translation ?
2. Convolution is convariant to translation ?

What is the difference between covariance and equivariance ?

1. Are convolutional layers equivariant to translation ?
• Does this come from any source? If so which? – Reviewer
– Jim
Jun 1, 2018 at 16:31
• aboveintelligent.com/… and and arxiv.org/abs/1804.03393 . I get confused at understanding the difference between covariance and equivariance Jun 1, 2018 at 16:38

There are two schools of thought when it comes to definitin of equivariance, covariance, invariance, and same-equivariance.

• Covariance is a concept often used in physics and is the same term as equivariance. Both are used when applying the transformation $$\pi$$ on the input of the function $$f$$ can be achieved by appying another transformation $$\psi$$ on the output of the function:
$$f(\pi(x))=\psi(f(x))$$

• Same-equivariance is an especial case of equivariance when $$\psi=\pi$$ (in some literature, same-equivariance is termed as equivariance, and instead, equivariance is termed covariance): $$f(\pi(x))=\pi(f(x))$$

• Invariance is another especial case when the transformation $$\psi$$ is the identity function ($$\psi=\mathbb{1}$$)
$$f(\pi(x))=f(x)$$

Based on the above definitions:

• convolution is "equivariant" to translation
• convolution is also "same-equivariant" to translation,
• and since covariance is just another term for the same concept, convolution is "covariant" to translation.

Same is true for convolutional layers.

• In two of your last three bullets, you said "convolution is equivariant to translation" twice; did you mean something else one of those times? Sep 11, 2022 at 9:42
• Thank you @xFioraMstr18. Fixed now.
– Amir
Sep 12, 2022 at 17:04