In deep learning papers, data augmentation is often presented as a type of regularization. For example, this is explored in Chiyan Zhang and coauthor's presentation at ICLR17, Understanding deep learning requires rethinking generalization. Why is this classification given? Intuitively, I see data augmentation as a way of expanding a dataset, but regularization as a means of modifying a training algorithm to (hopefully) improve the generalization error.
Regularization (traditionally in the context of shrinkage) adds prior knowledge to a model; a prior, literally, is specified for the parameters. Augmentation is also a form of adding prior knowledge to a model; e.g. images are rotated, which you know does not change the class label. Increasing training data (as with augmentation) decreases a model's variance. Regularization also decreases a model's variance. They do so in different ways, but ultimately both decrease regularization error.
Section 5.2.2 of Goodfellow et al's Deep Learning proposes a much broader definition:
Regularization is any modiﬁcation we make to a learning algorithm that is intended to reduce its generalization error but not its training error.
There is a tendency to asssociate regularization with shrinkage because of the term "l-p norm regularization"...perhaps "augmentation regularization" is equally valid, although it doesn't roll off the tongue.
User99889 had a good answer (+1), I will add some comments to elaborate the points.
The goal of regularization is reducing the Variance and increase Bias in Bias Variance Trade Off. This can be achieved in different ways, here are some examples:
- Increase the amount of the data, collect new data or derive new data from existing data / data augmentation.
- Reduce the complex of the model. For example, reduce the hidden layers or number of units in the hidden layer.
- Put constraints in the coefficients/parameters. For example, L1 or L2 regularization.
We will further to explain why User99889 mentioned about "add prior / domain knowledge". Let use MNIST data as an example, we can create more data by using the following knowledge:
- If we re-scale digit, the label will not be changed
- If we shift digit, the label will not be changed
Then, we have more data using domain knowledge and the model will have more Bias.