I get stuck at understanding the difference between invariance to translation, covariance to translation and equivariance to translation in the context of of convolutional neural network.

What does it mean :

  1. Convolution is equivariant to translation ?
  2. Convolution is convariant to translation ?

What is the difference between covariance and equivariance ?

  1. Are convolutional layers equivariant to translation ?
  • $\begingroup$ Does this come from any source? If so which? – Reviewer $\endgroup$
    – Jim
    Jun 1, 2018 at 16:31
  • $\begingroup$ aboveintelligent.com/… and and arxiv.org/abs/1804.03393 . I get confused at understanding the difference between covariance and equivariance $\endgroup$
    – vincet
    Jun 1, 2018 at 16:38

1 Answer 1


There are two schools of thought when it comes to definitin of equivariance, covariance, invariance, and same-equivariance.

  • Covariance is a concept often used in physics and is the same term as equivariance. Both are used when applying the transformation $\pi$ on the input of the function $f$ can be achieved by appying another transformation $\psi$ on the output of the function:

  • Same-equivariance is an especial case of equivariance when $\psi=\pi$ (in some literature, same-equivariance is termed as equivariance, and instead, equivariance is termed covariance): $f(\pi(x))=\pi(f(x))$

  • Invariance is another especial case when the transformation $\psi$ is the identity function ($\psi=\mathbb{1}$)

Based on the above definitions:

  • convolution is "equivariant" to translation
  • convolution is also "same-equivariant" to translation,
  • and since covariance is just another term for the same concept, convolution is "covariant" to translation.

Same is true for convolutional layers.

  • $\begingroup$ In two of your last three bullets, you said "convolution is equivariant to translation" twice; did you mean something else one of those times? $\endgroup$ Sep 11, 2022 at 9:42
  • $\begingroup$ Thank you @xFioraMstr18. Fixed now. $\endgroup$
    – Amir
    Sep 12, 2022 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.