Skip to main content
added 303 characters in body
Source Link
PlagTag
  • 133
  • 5

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states on page 3 after discussion of derivation:

..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

Edit: I read up a little now and think as the normalization is just like a copy process that does not change the number of parameters the author just makes this statement to ensure the benefit of this method. I first thought there is some twist here that makes the notion of capacity important.

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states on page 3 after discussion of derivation:

..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states on page 3 after discussion of derivation:

..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

Edit: I read up a little now and think as the normalization is just like a copy process that does not change the number of parameters the author just makes this statement to ensure the benefit of this method. I first thought there is some twist here that makes the notion of capacity important.

added 33 characters in body
Source Link
PlagTag
  • 133
  • 5

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states (pageon page 3) after discussion of derivation:

pa..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states (page 3):

pa, BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states on page 3 after discussion of derivation:

..., BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?

Source Link
PlagTag
  • 133
  • 5

Why is batch normalization preserving the capacity of a network?

I have a question regarding the Batch normalization paper of Sergey Ioffe

In the paper the author states (page 3):

pa, BN transform is a differentiable transformation that introduces normalized activations into the network. This ensures that as the model is training, layers can continue learning on input distributions that exhibit less internal covariate shift, thus accelerating the training. Furthermore, the learned affine transform applied to these normalized activations allows the BN transform to represent the identity transformation and preserves the network capacity

Question Why is this at all related to preserving the network capacity?