I'm looking into using Transfer Learning to take the ResNet50 model trained on ImageNet and fine tune it to my own dataset using Keras.

However, I feel I have some misconception about what exactly fine tuning is, and how to perform it.

In this paper, which I read many months back, I understood that transfer learning was a process where you took the first n layers from a pre-trained model, added on your own final layers for your task, and then fine tuning was where you did NOT freeze the weights from layers you transferred from the pre-trained model, but instead allowed them to update with a very low learning rate. I also understood that this method resulted in better generalisation, and results in general, than freezing the weights from the transferred layers.

However, every time I see fine tuning mentioned on the internet, people refer to freezing the weights from the transferred layers and applying a low learning rate to the new layers - only allowing their weights to update. As seen here.

This answer also recommends freezing the weights from the transferred layers.

I just don't see how this advice lines up with the results from the paper. It suggests that taking a large number of layers from the original network and freezing their weights will give a poor result, whereas allowing the weights from the transferred layers to be fine-tuned will improve generalisation.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.