I'm adding data augmentation on a FCN model, right now I'm doing simple flips, random zoom and random rotations. At the moment for each sample I do all the four transforms (vertical flip, horizontal flip, zoom, rotation) separately for each epoch.

So the dataset grows five times bigger, each epoch is about five times slower, but I get converge way faster than before.

Another alternative I've seen is to keep each epoch the same size of the original training set and for each example randomly sample from the augmented transform pool. I didn't try this approach but I guess it learns in more epochs in about the same time as the other one.

Is there a best practice about this?

The main problem, if I may call it so, I'm seeing with this approach is I get ugly metrics and loss plots with fewer data points as the number of epochs to convergence is small.


1 Answer 1


I think both are equally valid approaches, however, in practice, I have rarely seen your approach of doing all augmentations per epoch. I think most people just go with sampling from the augmentations.

Here are some reasons that might be the case

  1. Compatibility with augmentations such as random noise -- because looping over all possible random noise would be painful. Sampling also scales better when you just have a lot of transformations.

  2. It's much easier to make the mistake of not shuffling properly if you try to do all augmentations. For example, the following sequence, where the data is randomly shuffled, but the transforms are applied in order, is incorrect:

data1_transform1, data3_transform1, data_2_transform1, data1_transform2, data3_transform2, data_2_transform2, data1_transform3, data3_transform3, data_2_transform3

It would also not be ideal, but probably not too harmful, to do something like:

data1_transform1, data1_transform3, data1_transform2, data3_transform2, data3_transform1, data3_transform3, data2_transform3, data2_transform1, data2_transform2

As you can see, getting the shuffling exactly right is a bit tricky.

  1. Comparability to training without augmentation. It doesn't make sense to compare a network trained with augmentation for 1 epoch with a network which has been trained without augmentation for 1 epoch if the one which has, has also been through 5x as many iterations. So keeping the number of iterations per epoch constant is important for comparison.
  • $\begingroup$ I guess you're right I probably misinterpreted some paper that said they increased their dataset size x times with augmentation, and given an epoch is supposed to be a full dataset pass I thought the proper way was doing all the transforms for each epoch. That's gets tricky though when you start doing random transformations or adding noise. I'm almost sure I've seen it done this way though, I'll look for the source. $\endgroup$
    – filippo
    Nov 4, 2017 at 7:15
  • $\begingroup$ About shuffling, I'm not sure, if you consider each transformed example as a new example you just put everything in a list and shuffle that. Shuffling only the original examples or only transforms per example adds correlation $\endgroup$
    – filippo
    Nov 4, 2017 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.