4
$\begingroup$

In an article talking about ResNet, there has the following statement

The second, the bottleneck unit, consists of three stacked operations. A series of 1x1, 3x3 and 1x1 convolutions substitute the previous design. The two 1x1 operations are designed for reducing and restoring dimensions. This leaves the 3x3 convolution, in the middle, to operate on a less dense feature vector. Also, BN is applied after each convolution and before ReLU non-linearity.

I am not clear how to understand the statement of This leaves the 3x3 convolution, in the middle, to operate on a less dense feature vector.What does that mean? In specific, what does the less feature vector mean, what causes the generation of this less dense feature vector?

$\endgroup$
3
$\begingroup$

In the original ResNet paper (page 6), they have explained the use of these deeper bottleneck designs to build deep architectures.

Bottle Neck Unit

As you've mentioned these bottleneck units have a stack of 3 layers (1x1, 3x3 and 1x1). The 1x1 layers are just used to reduce (first 1x1 layer) the depth and then restore (last 1x1 layer) the depth of the input. Suppose the input has a depth of 256, then the first 1x1 layer can reduce the depth to 64 and the 3x3 convolution layer can then operate on this less dense (lower depth) feature vector. The final 1x1 layer will then restore it to its original depth (256). The main goal of this design is to increase the efficiency.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.