19
$\begingroup$

I am doing image classification using machine learning.

Suppose I have some training data (images) and will split the data into training and validation sets. And I also want to augment the data (produce new images from the original ones) by random rotations and noise injection. The augmentaion is done offline.

Which is the correct way to do data augmentation?

  1. First split the data into training and validation sets, then do data augmentation on both training and validation sets.

  2. First split the data into training and validation sets, then do data augmentation only on the training set.

  3. First do data augmentation on the data, then split the data into training and validation set.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ "Data augmentation" has more than one meaning; it'd help to edit your question to clarify which is yours, or just to give an example. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Oct 5, 2015 at 13:47
  • 1
    $\begingroup$ If you plan to do TTA, then the augmentation should be applied to the validation set as to the test set. $\endgroup$
    – abby yorker
    Commented Jul 30, 2019 at 20:50

3 Answers 3

Reset to default
28
$\begingroup$

First split the data into training and validation sets, then do data augmentation on the training set.

You use your validation set to try to estimate how your method works on real world data, thus it should only contain real world data. Adding augmented data will not improve the accuracy of the validation. It will at best say something about how well your method responds to the data augmentation, and at worst ruin the validation results and interpretability.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I am quite curious about something in your answer. If my criterium to stop training a CNN is reducing the validation loss, do you believe that data augmentation on the validation data is a good choice? $\endgroup$
    – mad
    Commented Mar 5, 2018 at 5:58
  • 2
    $\begingroup$ No, I still think that would "ruin the validation results and interpretability", as the validation accuracy is no longer a good proxy for the accuracy on new unseen data if you augment the validation data. $\endgroup$
    – burk
    Commented Mar 5, 2018 at 9:43
  • $\begingroup$ so we dont need to apply data augmentation on validation and testing data at all? $\endgroup$
    – Aadnan Farooq A
    Commented Jun 15, 2019 at 4:34
  • $\begingroup$ @AadnanFarooqA No. You should normally do the same operations on your testing and validation data as you intend to do on you unseen data when you use your model for predictions. $\endgroup$
    – burk
    Commented Jun 18, 2019 at 13:49
  • 1
    $\begingroup$ @AadnanFarooqA Normally you should just apply augmentation on the training data, after the split. $\endgroup$
    – burk
    Commented Jun 20, 2019 at 11:28
9
$\begingroup$

never do 3, as you will get leakage. for example assume the augmentation is a 1-pixel shift left. if the split in not augmentation aware, you may get very similar data samples in both train and validation.

Improve this answer
$\endgroup$
-1
$\begingroup$

Data Augmentation means adding external data/information to the existing data which is being analyzed.

So, as the entire augmented data would be used for machine learning, then the following process would be better suitable:

Do data augmentation --> Splitting data

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the reply. Is it all right that a sample and the augmented sample, which is quite similar to the original one, are spread in different sets? $\endgroup$
    – yangjie
    Commented Oct 5, 2015 at 11:31
  • $\begingroup$ You mean the existing data as a training set and the augmented data as a validation set? Then, NO $\endgroup$
    – Dawny33
    Commented Oct 5, 2015 at 11:36
  • $\begingroup$ The splitting is random, so I mean if I do data augmentation and then split the data, it is likely that some existing data (not all) is split into the training set, while the augmented data goes to the validation set. $\endgroup$
    – yangjie
    Commented Oct 5, 2015 at 12:17
  • $\begingroup$ By augmentation, do you mean appending? Augmented data is the data which supports current data at all points. So, if the splitting is random, then the split would result in the same amount of augmente data in both sets, as that of the existing data $\endgroup$
    – Dawny33
    Commented Oct 5, 2015 at 12:21
  • $\begingroup$ Is there any reference of paper for this? $\endgroup$
    – Aadnan Farooq A
    Commented Jun 14, 2019 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.