2
$\begingroup$

In the week 2 assignment of Dr Andrew Ng's Convolutional Neural Networks course, it says:

The identity block is the standard block used in ResNets, and corresponds to the case where the input activation (say $a^{[l]}$) has the same dimension as the output activation (say $a^{[l+2]}$).

enter image description here

In this exercise, you'll actually implement a slightly more powerful version of this identity block, in which the skip connection "skips over" 3 hidden layers rather than 2 layers. It looks like this:

enter image description here

  1. Why is skipping three layers "a slightly more powerful version" of the ResNet identity block? Is there any research on this?

  2. Is there an optimal number of layers to skip for various functions?

Improve this question
$\endgroup$
  • 1
    $\begingroup$ I think that it means that not the block contains 3 layers instead of just 2, thus being more powerful (not that skipping more layers is better). I'm pretty sure the number of layers to be skipped is empirical. If you're interested in skip connections I suggest you look into another very interesting architecture called "DenseNet". $\endgroup$ – Djib2011 Aug 4 '18 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.