Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have a pretrained net(vgg16) which is trained with Imagenet, where each input is standardized by subtract the mean of Imagenet dataset. Now i want fine tuning the net with another dataset (Parse27k), and my question is:
which mean i must subtract for the standardization during the fine tuning, the old one or the mean of the new dataset?
In order to obtain the best from the fine tuning process the new input data should be as similar as possible to the original used for the pre-training, this means, that if there is any pre-processing in the orignal model, then, the same preprocessing should take place in the new dataset.
I don't have any knowledge for the net vgg16, but, if they are substracting the mean of the whole dataset, then you should do the same, substract the mean of the previous dataset.
I would recommend you to do 3 experiments, substracting the mean of the original dataset, of the new one, and doing nothing, you will notice that the network will train faster and better in the first experiment.
Another experiment you can make is to restore the weights of all the layers and include another layer in the beginning, just a bias so the network will decide the best standarization parameter
