How to perform data augmentation with traditional machine learning algorithms?

Question

I am currently working on a multi-class image classification project, in which I have to use traditional machine learning and feature extraction methods (no convolutional neural networks). I know data augmentation is basic step for CNNs, allowing the network to generalize as, at each batch, images are loaded with different transformations (as flipping, rotation, zooming, etc.). However, I am having troubles to imaging it applied to traditional ML algorithms such as SVM or KNN, as they receive all the training examples at once, so these data are in a certain way "static".

Let's say I have 1000 training images. At this point, I have doubled my training dataset, training my model on both original and flipped images (for a total of 2000 images). What should I do if I also want to use more augmentation techniques, e.g. rotations? Should I further duplicate training images with random rotation (reaching size 3000 or higher), or instead rotate the ones I already have (keeping size 2000), always avoiding overfitting?

Thanks in advance.

Answer 1

There's multiple ways to use (image) augmentation with different machine learning approaches:

• Approaches that update model parameters in iterations (aka epochs for neural networks): In these, you can for each iteration (or epoch) change the training data e.g. using data augmentation, gradual resizing and so on. Example of algorithms, where this fits, include neural networks, SVMs, most standard regression models and so on. Of course, standard implementations of these algorithms do not typically provide facilities for doing things (except for neural networks).
• Tree based approaches (e.g. random forest or gradient boosted trees): In these, one can augment the data differently for each tree (or even for each node in a tree). Again, standard implementations of these algorithms do not provide facilities for doing this.
• One can also augment the training data multiple times and combine these augmented copies of the training data with the original data. This strategy works with just about any machine learning algorithm, but of course will make the dataset substantially larger so that one pass through it will take longer (possibly an issue with e.g. SVMs). The important thing though is that you end up with non-independent records and you need to be careful:
  • E.g. you need to make sure that test- and validation-splits (including for cross-validation) never split different versions of the same image into different parts of a split. Otherwise, you will mis-evaluate performance (often very, very badly) and end up setting the hyperparameters you pick (e.g. via cross-validation) completely wrongly (the algorithm would be rewarded for managing to re-recognize augmented version of an image, instead of "learning" something that generalizes).
  • This usually means manually creating your own (cross-)validation and test splits instead of just relying on some standard parameter for number of validation folds in a call to a model function. Luckily, most standard implementations of models (e.g. scikit-learn etc.) make it easy to specify your own splits.
  • Additionally, you'd ideally want algorithms that involve sampling records or boostrapping them to respect what images are augmented versions of each other. I'm not aware of anything that does this other than for neural networks. I'm also not entirely sure how much this hurts performance, but it probably does a bit.

As you can see, there are many options and some are easier to implement with out-of-the-box functions/packages than others, while some approaches would likely involve writing your own implementation of some algorithms.

By the way, if you are just not meant to use convolutional neural networks, then there are of course other neural networks that perform about as well, where image augmentation and transfer learning are, again, easy. These include image transformers and multi-layer perceptrons (see e.g. the models covered in the PyTorch image models package).

Answer 2

Lets have a look at wikipedia.

Data augmentation in data analysis are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data.

If you were only to modify your data instead of combining your orginal data with the modified one, you would not be augmenting your dataset. Instead you would merely be preprocessing images.

If you wish to augment your dataset, which often makes sense with images (after all, a car is a car even if sligtly moved to the right), then do combine them.

Further, I do not see how such augmentation should be limited to neural networks.

Note that you can even stack modifications. Given a single image, you could then have:

• the original
• the flipped original
• the rotated orginal
• the flipped rotated orginal
• the rotated flipped original

But be aware that some of the augmentation methods do not make any, depending on the dataset (example: rotate a 6 in the EMNIST dataset to become a 9)