I was trying to understand how the contributions of Residual Nets differed from batch normalization. I have read both papers but its still not clear to me.
As far as I can tell batch normalization essentially solved the issue of vanishing and exploding gradients. However, intuitively for me it seems that in principle this was mainly caused by depth of a network. Thus, why is it that batch normalization is not able to train networks that are as deep as residual networks? What is special about ResNets that batch norm does not do?
How do the contributions of ResNets and batch norm differ? Is it that skip connection can learn the identity better and thus much deep nets are trainable? Do ResNets not work without batch norm meaning ResNets do not solve the vanishing/exploding gradient problem at all?
How do the contributions differ?