How effective is randomly/algorithmically generating exorbitant amounts of training data for a neural network?

Question

There are some problems of which the generating of data is easy whereas the inverse is not.

For example, use a 3D game engine to render some randomly generated objects with some random changes and place them throughout the scene. We already know the positions of these objects so these are the "answers" when training the neural net, and the "input" would be the rendered pixels at a particular viewpoint.

For example, use a speech synthesis algorithm with randomized formant, pitch, EQ, speed, random reverb/noise/distortion, as well as randomized words as input. The "answer" is the input to the speech synthesis algorithm, and the "input" to the neural net is the output of the speech synthesis algorithm.

For example, in the "blind source separation" problem, pick a ton of sounds from Youtube, or freesound, and randomly add them together in resulting sound files (preferably each source audio would have a single source in real life, like one person's voice). The resulting summed cacophonies will be training data, and the original signals are ground truth.

Is there a name to technique? What is it called? Has it ever been implemented successfully without overfitting? What kind of overfitting problems would result?

Answer 1

This is a pretty common technique for computer vision; people often rotate images, add random noise, flip them, zoom in, zoom out, and add random shifts in order to add lots of extra training data for deep convolutional networks. I've seen it called data augmentation or data percolation. I don't know for sure, but I could imagine it would be useful in other fields such as speech recognition too.

Answer 2

I suggest the name learning from synthetic data as used here and I also recommend you to read about domain adaptation because you will likely need it when you train on such synthetic data.

While synthetic data are cheap to label, the joint distribution of observed synthetic data and their corresponding target variables is still not the same as the corresponding distribution over the testing data. Thus you will need to adjust your learned models somehow (or adjust vector representation of your synthesized training data) so that your resultant model adapts better to the testing set (target problem set) -- this is domain adaptation.