If I take Fig.3 of the paper "Deep residual learning for image recognition", and look at the following piece of the residual network:

$3\times3$ conv, 64 filters

   | (X) (suppose shape is 14*14*64

$3\times3$ conv, 128 filters, stride=2

   | (X') (shape will be 7*7*128

$3\times3$ conv, 128 filters

   v (F(X) (shape will be 7*7*128)

I thus must sum (element-wise) the result $X + F(X)$ which are of different shapes. However, while a $1\times1$ convolution can help to get the same depth (number of features), what is the traditional way to obtain the same width/height for both X and F(X)?

Should I compute MaxPooling(X, stride=2) + F(X)?


3 Answers 3


The default way of solving this is to a 1x1 conv with a stride of 2, followed by a batch norm. Yes, half of the pixels will be ignored.

See the implemention in pytorch (from FAIR where the authors work): link


I happened to read this paper recently. This paper introduced a shortcut projection to match the dimension. You can see equation 2 in the paper. Three different projection options A, B, C is compared in page 6.

There is a tensorflow implementation of residual net. You can find implementation of shortcut projection in real code.

Good luck!


I'm running into the same issue. It looks like there are two ways to solve this: "When the input and output dimensions don’t match up, we add a convolutional layer in the shortcut path. The arrangement is called convolutional block" https://engmrk.com/residual-networks-resnets/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.