2
$\begingroup$

enter image description here

In the above image, It is the part of the resNet Architecture, here they have used dotted line to increase the dimension, but my question is How they are increasing the dimension?? or this dotted line is just a convolution layer to increase the dimension?

$\endgroup$
4
$\begingroup$

That essentially means either linear skip connection, or padding $\mathbf{x}$ (input to the residual block) to appropriate shape.

Note that in equation $(2)$ of the ResNet paper:

$$\textbf{y} = \mathcal{F}(\textbf{x}, W_i) + W_s\textbf{x}$$

You can have $W_s$ mapping $\textbf{x}$ to the desired space.

Here is the excerpt on this from the paper:

When the dimensions increase (dotted line shortcuts in Fig. 3), we consider two options:

(A) The shortcut still performs identity mapping, with extra zero entries padded for increasing dimensions. This option introduces no extra parameter;

(B) The projection shortcut in Eqn.(2) is used to match dimensions (done by 1×1 convolutions).
For both options, when the shortcuts go across feature maps of two sizes, they are performed with a stride of 2.

$\endgroup$
4
  • $\begingroup$ Thanks for your quick reply, Yes I have understood the part of the padding,ok, In the (B) they have just used (1x1) convolution to increase the feature map, am I right? $\endgroup$
    – Sudip Das
    Feb 24 '18 at 9:02
  • $\begingroup$ Yes, that is correct. $\endgroup$ Feb 24 '18 at 11:27
  • $\begingroup$ @JakubBartczuk how does 1x1 convolution increase the size? It would keep same! It would help balance out the number of channels but what if HxW is different? Then we would need (A) right? $\endgroup$ Jul 8 '20 at 5:05
  • $\begingroup$ It seems that a 1x1 with stride > 1 is used to reduce height and width despite ignoring completely some neurons. At least, this is what I read online. $\endgroup$ May 5 '21 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.