I have a question regarding the Batch normalization paper of Sergey Ioffe
In the paper the author states on page 3 after discussion of derivation:
..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity
Question Why is this at all related to preserving the network capacity?
Edit: I read up a little now and think as the normalization is just like a copy process that does not change the number of parameters the author just makes this statement to ensure the benefit of this method. I first thought there is some twist here that makes the notion of capacity important.