CNN: ReTraining and Fine Tuning I am getting into CNNs for several months already, and I am currently wondering myself if there is a difference between Retraining a DCNN and Fine Tuning it ?
I am working on a project in which my own new dataset is relatively small and very different from the original one. Then I don't really understand new what I have to look for.
 A: Retraining a DCNN on another data set = Fine Tuning it 
http://cs231n.github.io/transfer-learning/ (mirror):

Fine-tuning the ConvNet. The second strategy is to not only replace and retrain the classifier on top of the ConvNet on the new dataset, but to also fine-tune the weights of the pretrained network by continuing the backpropagation. It is possible to fine-tune all the layers of the ConvNet, or it’s possible to keep some of the earlier layers fixed (due to overfitting concerns) and only fine-tune some higher-level portion of the network. This is motivated by the observation that the earlier features of a ConvNet contain more generic features (e.g. edge detectors or color blob detectors) that should be useful to many tasks, but later layers of the ConvNet becomes progressively more specific to the details of the classes contained in the original dataset. In case of ImageNet for example, which contains many dog breeds, a significant portion of the representational power of the ConvNet may be devoted to features that are specific to differentiating between dog breeds.

