I find the documentation and tutorials on the Internet surrounding ImageDataGenerator (the data augmentation function for Keras) to not really explain much at all how it works.

Following the instructions here also leads room for a lot of interpretation: https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

I have two questions. First, am I supposed to run model.fit_generator in addition to the normal model.fit? If so, how do I do this? When I try running model.fit_generator and then model.fit I find my validation loss increases (and accuracy decreases) when running model.fit after model.fit_generator. If I'm not supposed to use model.fit at all, wouldn't this be ignoring the non-augmented training set data?

Secondly, I don't understand the example in the above blog post where we input a validation generator for the model.fit_generator instead of using non-augmented data for the validation set. The entire point of the validation set is to see how the model predicts real world data, not how well it can predict synthetic augmentation data that is of no use. So then, what is the point of augmenting the validation set? How well the model predicts augmented data tells me nothing about how well it can predict real-world data, and I thought this augmented data would only be useful for the training set, not the validation set.



closed as off-topic by Tim Jan 9 '18 at 16:57

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Tim
If this question can be reworded to fit the rules in the help center, please edit the question.


First, am I supposed to run model.fit_generator in addition to the normal model.fit?

Just model.fit_generator will do the job. Calling model.fit_generator then model.fit will do the training twice with and without augmentation.

If I'm not supposed to use model.fit at all, wouldn't this be ignoring the non-augmented training set data?

Right, except that there's a (small) probability the generator will just give you the non-augmented data by chance. So if you want to mix augmented and non-augmented data, you might need some extra work.

So then, what is the point of augmenting the validation set?

There're many reasons for doing test time augmentation, for example if you train a network for detecting faces in images at some certain scale, and at test time you want to deal with images of different resolutions, you can augment the images by different scales to be able to detect faces at all scales.

Also this paper and this paper show that augmentation can lead to better performance on image recognition/segmentation tasks.

  • $\begingroup$ Thanks for the info. Can I ask should the test/live images then be augmented as per the example you gave, or is it only the training set we need to augment for greatest accuracy? $\endgroup$ – user4779 Apr 5 '17 at 12:38
  • 1
    $\begingroup$ @user4779 for image recognition tasks I found doing both training and test time augmentation will give the best performance as claimed in that paper. and I guess it should work for other tasks in general as long as the augmentation doesn't change the information of the data wrt the task. $\endgroup$ – dontloo Apr 5 '17 at 13:02
  • $\begingroup$ Thank you again! Are you aware if the augmentation will occur automatically when using model.predict from a model trained with model.fit_generator? If not, would I need to create a separate generator to augment each live image input? $\endgroup$ – user4779 Apr 6 '17 at 4:43
  • $\begingroup$ @user4779 you're welcome, I don't think model.predict will call the generator automatically, using a separate generator seems the right thing to do, because at test time we normally don't want the randomness introduced at training time. $\endgroup$ – dontloo Apr 6 '17 at 8:49

Alright, TL;DR:

  1. You shouldn't call model.fit after model.fit_generator
  2. No, validation data is not the measure on real world data.

More in depth answer:

  1. The generator you create IS the data augmentation, if you do model.fit you probably have non-augmented data as X. This will probably overfit very quickly.
  2. Data is usually split into two chunks training-data and test-data. Afterwards a validation-data is chosen as a subset of training-data. Sometimes validation-data is also non-intersecting with training-data. The metric calculation on huge datasets is costly and using only a fraction (the validation-data) as the target, you get faster training, because less time is spent trying to aggregate feedback metric (such as precision). What you probably want, is to have training and validation generators constructed after train/test data split and at the end of training get test precision.

Call fit_generator doesn't mean you augment the data. In the test_generator the only thing it does is rescale the image. So no augmentation is done on validation set.


Not the answer you're looking for? Browse other questions tagged or ask your own question.