Tensorflow Inception: Accuracy of transfer learning vs training from scratch

Question

For those of you who have used Tensorflow's pre-trained Inception model to bootstrap an image classifier with custom categories and also trained a classifier from scratch with the same images, how big was the difference in validation/test accuracy between the two? If training from scratch yielded higher accuracy, how many training images did you have per class?

I need to build several custom classifiers and am trying to gage whether it's worth the effort and cost of an AWS GPU instance to train from scratch. The classifiers each have between 5 and 20 classes, with 500 - 5000 training images per class. I'm getting accuracy in the low 80% range for each classifier, and I'm not sure I can do much better because it's a very fine-grained classification task, but I'd like to try if someone can report a dramatic improvement after training from scratch.

Answer 1

how big was the difference in validation/test accuracy between the two?
The performance of the fine-tuned CNN depends upon the size of your dataset and the similarity between your dataset and dataset on which pre-trained CNN was trained. So there is no general pattern in the difference in validation/test accuracy between the two.

If training from scratch yielded higher accuracy, how many training images did you have per class?
The dataset having 500 - 5000 training images per class is definitely not sufficient to train large models like inception from scratch.

To improve the performance of a pre-trained CNN, you should try to backprop till last 2-3 inception block during training on pre-trained inception model.

For More detail : Transfer Learning