I am trying to implement the paper Image Style Transfer Using Convolutional Neural Networks. In section 2 - Deep image representations, the authors mention the following weight normalization technique:
We normalized the network by scaling the weights such that the mean activation of each convolutional ﬁlter over images and positions is equal to one. Such re-scaling can be done for the VGG network without changing its output, because it contains only rectifying linear activation functions and no normalization or pooling over feature maps.
From a related question asked earlier I found that the authors are using the activation values from images of ILSVRC validation set to normalize the weights.
I wanted to know the mathematical formulation of performing such a normalization as I was unable to come up with one myself.
According to my understanding of the problem I have an activation map (X) and corresponding to it I have K activation maps of the previous layer(L) and a weight matrix (W) of dimensions 3x3xK, such that when layer L is convolved with W it produces X. Now, once I captured the activation values for all the neurons in layer L for all images in the validation set, the objective is to make the mean of all neurons in X across all images in the validation set equal to 1 by somehow adjusting W.
I could not figure out what should I do to W to make that happen.
Also, I wanted to know if this is to be performed in a cascaded(sequential) way by normalizing weights of the initial layer first and then using the new feature maps to normalize the weights of the layers ahead or independently for each activation map by taking the values of previous layer as the original pre-trained weights for every activation map?