Why is using keras ImageDataGenerator for data augmentation relevent? I have used keras ImageDataGenerator to generate more data in my neural networks as I have had really small datasets and it has proven itself. As far as I understand, it generates new batches each epoch on the fly and so the neural network is trained on a "little bit" new data on each iteration.
Why/how is this method relevant? Is there any research done on this topic? I just haven't seen it in my studies so far and want to know whether this is commonly used method or just some feature that keras offers?
 A: Yes, there's sophisticated literature base on this topic. At this point, I think it's safe to say that that transforming image inputs is widely accepted as a way to improve the robustness of a network, so the practical question becomes which transformations are best, and how to cheaply go about generating good transformations for particular use-cases.
The basic idea is that if you're doing an image classification task, the network should do a good job of detecting the same object even if it's positioned differently within the frame (translated, rotated, etc). In the most general setting, the purpose is to simulate the fact that there's no particular need for the photographer to have taken a picture with a specific composition or arrangement of the objects. The same reasoning applies to varying brightness and other image attributes. 
Intuitively: A dog is a dog, whether the photo is taken head-on, from above, or from the side, or if the dog is illuminated by dim or bright light.
What augmentations are "admissible" depend on your goals. Not all augmentations align with specific tasks, so training on irrelevant modifications are unlikely to improve your classifier. For example, if you translate an image too much and the target object falls out of frame, then there's no longer relevant semantic content to classify.
Some papers discussing transformations of images to improve neural networks:


*

*"Deep Convolutional Inverse Graphics Network"
Tejas D. Kulkarni, Will Whitney, Pushmeet Kohli, Joshua B. Tenenbaum includes transformations of images as a step.

*Perusing the top results for CIFAR-10 reveals that data augmentation is often a step in the pipeline.

*Augmentation can even be unsupervised, which can be important if labels are scarce or expensive. Example: "Unsupervised Data Augmentation" by Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, Quoc V. Le.
