For those of you who have used Tensorflow's pre-trained Inception model to bootstrap an image classifier with custom categories and also trained a classifier from scratch with the same images, how big was the difference in validation/test accuracy between the two? If training from scratch yielded higher accuracy, how many training images did you have per class?
I need to build several custom classifiers and am trying to gage whether it's worth the effort and cost of an AWS GPU instance to train from scratch. The classifiers each have between 5 and 20 classes, with 500 - 5000 training images per class. I'm getting accuracy in the low 80% range for each classifier, and I'm not sure I can do much better because it's a very fine-grained classification task, but I'd like to try if someone can report a dramatic improvement after training from scratch.