# How do the residual blocks prevent exploding gradients?

I am reading Roger Grosse's lecture notes on ResNet and I have a question regarding the explanation on how residue blocks prevent gradient explosion, see the screenshot below:

My confusion is: this seems to only explain how the gradients won't vanish. On one hand $\partial F / \partial x$ is small as assumed there, the gradient would not explode anyway without the identity matrix. On the other hand, if the partial derivative is not small, the identity matrix is not going to save potential explosions.

So can someone clarify this explanation? Thanks in advance.

• Welcome to CV! On this site there's no need to say "thank you" at the end of your post - it might seem rude at first, but it's part of the philosophy of this site (tour) to "Ask questions, get answers, no distractions" and it means future readers of your question don't need to read through the pleasantries. May 14 '18 at 15:11

You say if $\partial F / \partial x$ was small then the gradient wouldn't explode. However, it would vanish towards 0 in the nonresidual case, which is also problematic. On the other hand, it becomes close to the identity matrix in the residual case, which is good.