6
$\begingroup$

I am reading this paper http://zli115.web.engr.illinois.edu/wp-content/uploads/2016/10/0479.pdf

It distinguishes between feature extraction and fine tuning in deep learning. I am not getting the difference as feature extraction is just the same as fine tuning:

As per my understanding:

You train a model on a dataset, use it for training on another dataset. This is fine tuning. This is the same as feature extraction from the first trained model, like in feature extraction also you take the first model and train it on a new dataset.

Is there any difference between the two in the ml literature?

Joint training is a third category I understand as there you train on all data simultaneously.

$\endgroup$
9
$\begingroup$

As shown in figure 2 of {1}, in the fine-tuning strategy all weights are changed when training on the new task (except for the weights of the last layers for the original task), whereas in the feature extraction strategy only the weights of the newly added last layers change during the training phase:

enter image description here


References:

$\endgroup$
  • $\begingroup$ I wonder why they call it feature extraction, better call it feature freezing. thanks for the explanation. $\endgroup$ – Rafael Jan 9 '17 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.