I'm adding data augmentation on a FCN model, right now I'm doing simple flips, random zoom and random rotations. At the moment for each sample I do all the four transforms (vertical flip, horizontal flip, zoom, rotation) separately for each epoch.
So the dataset grows five times bigger, each epoch is about five times slower, but I get converge way faster than before.
Another alternative I've seen is to keep each epoch the same size of the original training set and for each example randomly sample from the augmented transform pool. I didn't try this approach but I guess it learns in more epochs in about the same time as the other one.
Is there a best practice about this?
The main problem, if I may call it so, I'm seeing with this approach is I get ugly metrics and loss plots with fewer data points as the number of epochs to convergence is small.