I am reading Roger Grosse's lecture notes on ResNet and I have a question regarding the explanation on how residue blocks prevent gradient explosion, see the screenshot below:
My confusion is: this seems to only explain how the gradients won't vanish. On one hand $\partial F / \partial x$ is small as assumed there, the gradient would not explode anyway without the identity matrix. On the other hand, if the partial derivative is not small, the identity matrix is not going to save potential explosions.
So can someone clarify this explanation? Thanks in advance.