I am doing image classification using machine learning.

Suppose I have some training data (images) and will split the data into training and validation sets. And I also want to augment the data (produce new images from the original ones) by random rotations and noise injection. The augmentaion is done offline.

Which is the correct way to do data augmentation?

  1. First split the data into training and validation sets, then do data augmentation on both training and validation sets.

  2. First split the data into training and validation sets, then do data augmentation only on the training set.

  3. First do data augmentation on the data, then split the data into training and validation set.

  • 1
    $\begingroup$ "Data augmentation" has more than one meaning; it'd help to edit your question to clarify which is yours, or just to give an example. $\endgroup$ – Scortchi - Reinstate Monica Oct 5 '15 at 13:47
  • $\begingroup$ If you plan to do TTA, then the augmentation should be applied to the validation set as to the test set. $\endgroup$ – abby yorker Jul 30 '19 at 20:50

First split the data into training and validation sets, then do data augmentation on the training set.

You use your validation set to try to estimate how your method works on real world data, thus it should only contain real world data. Adding augmented data will not improve the accuracy of the validation. It will at best say something about how well your method responds to the data augmentation, and at worst ruin the validation results and interpretability.

  • $\begingroup$ I am quite curious about something in your answer. If my criterium to stop training a CNN is reducing the validation loss, do you believe that data augmentation on the validation data is a good choice? $\endgroup$ – mad Mar 5 '18 at 5:58
  • 1
    $\begingroup$ No, I still think that would "ruin the validation results and interpretability", as the validation accuracy is no longer a good proxy for the accuracy on new unseen data if you augment the validation data. $\endgroup$ – burk Mar 5 '18 at 9:43
  • $\begingroup$ so we dont need to apply data augmentation on validation and testing data at all? $\endgroup$ – Aadnan Farooq A Jun 15 '19 at 4:34
  • $\begingroup$ @AadnanFarooqA No. You should normally do the same operations on your testing and validation data as you intend to do on you unseen data when you use your model for predictions. $\endgroup$ – burk Jun 18 '19 at 13:49
  • 1
    $\begingroup$ @AadnanFarooqA Normally you should just apply augmentation on the training data, after the split. $\endgroup$ – burk Jun 20 '19 at 11:28

never do 3, as you will get leakage. for example assume the augmentation is a 1-pixel shift left. if the split in not augmentation aware, you may get very similar data samples in both train and validation.


Data Augmentation means adding external data/information to the existing data which is being analyzed.

So, as the entire augmented data would be used for machine learning, then the following process would be better suitable:

Do data augmentation --> Splitting data

  • $\begingroup$ Thanks for the reply. Is it all right that a sample and the augmented sample, which is quite similar to the original one, are spread in different sets? $\endgroup$ – yangjie Oct 5 '15 at 11:31
  • $\begingroup$ You mean the existing data as a training set and the augmented data as a validation set? Then, NO $\endgroup$ – Dawny33 Oct 5 '15 at 11:36
  • $\begingroup$ The splitting is random, so I mean if I do data augmentation and then split the data, it is likely that some existing data (not all) is split into the training set, while the augmented data goes to the validation set. $\endgroup$ – yangjie Oct 5 '15 at 12:17
  • $\begingroup$ By augmentation, do you mean appending? Augmented data is the data which supports current data at all points. So, if the splitting is random, then the split would result in the same amount of augmente data in both sets, as that of the existing data $\endgroup$ – Dawny33 Oct 5 '15 at 12:21
  • $\begingroup$ Is there any reference of paper for this? $\endgroup$ – Aadnan Farooq A Jun 14 '19 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.