7
$\begingroup$

Is it common practice to apply data augmentation to training set only, or to both training and test sets?

$\endgroup$
  • $\begingroup$ I thought test time augmentation is pretty common these days. I first read about it in the 2012 paper here: papers.nips.cc/paper/… $\endgroup$ – Ryan Zhang Jan 9 at 17:10
6
$\begingroup$

In terms of the concept of augmentation, ie making the data set bigger for some reason, we'd tend to only augment the training set. We'd evaluate the result of different augmentation approaches on a validation set.

However, as @Łukasz Grad points out, we might need to perform a similar procedure to the test set as was done on the training set. This is typically so that the input data from the test set resembles as much as possible that of the training set. For example, @Łukasz Grad points out the example of image cropping, where we'd need to crop the test images too, so they are the same size as the training images. However, in the case of the training images, we might use each training image multiple times, with crops in different locations/offsets. At test time we'd likely either do a single centred crop, or do random crops and take an average.

Running the augmentation procedure against test data is not to make the test data bigger/more accurate, but just to make the input data from the test set resemble that of the input data from the training set, so we can feed it into the same net (eg same dimensions). We'd never consider that the test set is 'better' in some way, by applying an augmentation procedure. At least, that's not something I've ever seen.

On the other hand, for the training set, the point of the augmentation is to reduce overfitting during training. And we evaluate the quality of the augmentation by then running the trained model against our more-or-less fixed test/validation set.

$\endgroup$
  • $\begingroup$ I think it should be @Łukasz Grad instead of me in the above $\endgroup$ – MachineEpsilon Dec 31 '17 at 4:44
1
$\begingroup$

Typically, data augmentation for training convolutional neural networks is only done to the training set. I'm not sure what benefit augmenting the test data would achieve as the value of test data is primarily for model selection and evaluation and you're adding noise to your measurement of those quantities.

$\endgroup$
  • $\begingroup$ I diasagree, eg. most papers using imagenet dataset train and test their classifier with random cropping, which is a form of augmentation $\endgroup$ – Łukasz Grad Dec 31 '17 at 2:20
  • $\begingroup$ I certainly could be wrong, do you mind providing a reference? A quick sample of some papers like AlexNet nvidia.cn/content/tesla/pdf/machine-learning/…, Resnet arxiv.org/pdf/1512.03385.pdf, and YOLO9000 arxiv.org/pdf/1612.08242.pdf and it seems like none of these do augmentation on the test set (so far as I can tell). $\endgroup$ – MachineEpsilon Dec 31 '17 at 2:41
  • 1
    $\begingroup$ In a sense I think you're both right: if a net was trained with random crop, the test images will tend to be cropped too. But they might not be a random crop: they might be a centre crop. But not always. I'm not really sure this is 'augmentation' of the test set as such, so much as ensuring the distribution of the input data in the test set somewhat matches that of the training set. But that's semantics really: from a technical point of view, one might need to do something to the test set so that it resembles the training set, similar to how dropout works at test time. $\endgroup$ – Hugh Perkins Dec 31 '17 at 2:55
  • 1
    $\begingroup$ Yes, that makes sense. As best as I can see cropping is a special case because it effects the model architecture by changing the size of the input layer and other augmentation transformations (such as adding noise, reflections, blurring) do not. $\endgroup$ – MachineEpsilon Dec 31 '17 at 4:52
  • 1
    $\begingroup$ @Machineepsilon Here is the first example I could find from inception pape, table 4: arxiv.org/pdf/1512.00567.pdf $\endgroup$ – Łukasz Grad Dec 31 '17 at 9:09

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.