1
$\begingroup$

I have a pretrained net(vgg16) which is trained with Imagenet, where each input is standardized by subtract the mean of Imagenet dataset. Now i want fine tuning the net with another dataset (Parse27k), and my question is:

which mean i must subtract for the standardization during the fine tuning, the old one or the mean of the new dataset?

$\endgroup$
1
$\begingroup$

In order to obtain the best from the fine tuning process the new input data should be as similar as possible to the original used for the pre-training, this means, that if there is any pre-processing in the orignal model, then, the same preprocessing should take place in the new dataset.

I don't have any knowledge for the net vgg16, but, if they are substracting the mean of the whole dataset, then you should do the same, substract the mean of the previous dataset.

I would recommend you to do 3 experiments, substracting the mean of the original dataset, of the new one, and doing nothing, you will notice that the network will train faster and better in the first experiment.

Another experiment you can make is to restore the weights of all the layers and include another layer in the beginning, just a bias so the network will decide the best standarization parameter

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.