# How does the Identity connection in ResNets work

I am currently going through the Research paper "Deep Learning for Image recognition" by Kaiming He. I don't quite understand the concept of shortcut connections. Suppose the input to the residual learning layer shown below is an image of size 32x32. As per my understanding, after the two convolution layers shown in the figure the dimension of the output will decrease to 28x28.

How is then the element wise addition of this 28x28 image and the 32x32 input image possible?

Is the convolution mentioned in this layer not the same as in CNN?

The 3x3 convolution is performed with a zero-padding of 1 layer, so that the dimensions remain unchanged.

ie., 32x32 -> zero-padding-> 34x34 -> 3x3 conv -> 32x32

• Thank you very much. So what is meant by identity connection please?
– Avv
Commented Mar 19, 2023 at 20:30

Newstein already answered your question, but even if dimensions of conv output and $x$ are different, they say the following (section 3.2 of the paper):

"The dimensions of $x$ and $F$ must be equal in Eqn.(1). If this is not the case (e.g., when changing the input/output channels), we can perform a linear projection $W_s$ by the shortcut connections to match the dimensions".

• actually, shortcut connection is a (1x1) convolution layer, my question is that do i need to use activation function in that shortcut connection? Commented Mar 15, 2018 at 8:57
• Could you please explain further? What does Ws exactly denote? Are these learnable parameters? If so, would that not go against the very idea of identity mappings? How does Ws change the dimensions and still preserve the input data? Commented Nov 8, 2019 at 5:02
• Thank you very much. So what is meant by identity connection please?'
– Avv
Commented Mar 19, 2023 at 20:30