I was going through a brief tutorial on "transfer learning" available on https://www.analyticsvidhya.com/blog/2017/06/transfer-learning-the-art-of-fine-tuning-a-pre-trained-model/
In this blog, a distinction has been made between training the initial layers and training the dense layers of a neural network. The author highlights that when the data similarity is less between two domains, it is better to fine tune the lower layers of neural network while it is more efficient to train the dense layers of neural network when the data similarity is significant.
In this context, could anyone explain to me the logistics that governs the use of one strategy against the other?
Typically, I want to know in what way will the final model behave differently when the initial layers are pretrained versus when the dense layers are pretrained?