Can a GAN be used for data augmentation?

Question

Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model trained on the augmented dataset?

Answer 1

Yes, GAN can be used to "hallucinate" additional data as a form of data augmentation.

See these papers which do pretty much what you are asking:

• Data Augmentation Generative Adversarial Networks
• Low-Shot Learning from Imaginary Data
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

If your GAN is sufficiently well trained, there's no reason why this shouldn't help improve model performance. If your GAN is bad, you'll get garbage.

Answer 2

If you train your GAN on dataset A and use it to augment data on B, I think the answer is yes, since it absorb some knowledge from A. If you train your data on B and try to augment on B, I think GAN is useless here because there is no gain in information.

Answer 3

As already mentioned, it depends on what data you train your GAN. But it also depends on what you expect as an outcome of the GAN. Most methods focus on complete new synthetic data, but that's not the only option. This approach of landing AI seems to be more promising than just generating new artificial data, they augment existing data using a GAN to enrich it with rare events. This can help to generate data for classes that are underrepresented in the training data or could help to mitigate bias in some cases.

Answer 4

(My answer will mostly focus on tabular data as it has proven the hardest to synthesize owning to its heterogeneity and general arbitrariness)

Yes, we can get a generative adversarial network (GAN) to generate synthetic data. An exceptional resource on this is: the Synthetic Data Vault initiative where a number of different approaches regarding synthetic data generation are included. As you correctly assess, GANs can be used for synthetic data generation, a number of approaches are implemented in the accompanying sdv package. I will note here that actually variational auto-encoders (VAEs) seem to be a very competitive alternative to GANs for this task. The last couple of years there have been quite a good papers on the subject, some obvious picks would be:

• Modeling Tabular data using Conditional GAN (2019) by Xu et al.
• Data Synthesis based on Generative Adversarial Networks (2018) by Park et al.
• Learning vine copula models for synthetic data generation (2018) by Sun et al.

From personal experience: Synthetic data generation can be exceptionally hard to get "arbitrarily correct" but it can be possible to get reasonable mileage out it. For example: Making a synthetic dataset where a classifier cannot distinguish between original and synthetic data is very hard. Making a synthetic dataset to pass to collaborators for them to train some classifier such that it performs adequately in unseen real data is just hard. Making a synthetic dataset to pass to a collaborator to make some EDA and presentation material is usually doable. As such we need to consider why we need the synthetic data to begin with:

• is it for a privacy preserving task?
• is it a data-scarcity issue?
• is it for model-testing requirements? (common in financial applications - eg. stress testing in banking)

(Other, more exotic use cases, also exist: complicated data-sharing agreements, adversarial learning applications, agent-based modelling, etc. but they are rarer beasts.)

These points are not particular to tabular data but rather extend to all types of data. For example, one healthcare trust might want/have to avoid sharing real patient MRI scans (imaging data / privacy preserving scenario), similarly a tech start-up might need to train its voice recognition software on multiple speakers (speech data / data-scarcity scenario). Notably even GANs models with strong theoretical foundations (Wassenstein GAN (2017) Arjovsky et al.) might not be fully appropriate, for example financial time-series data have long-term correlations, volatility clustering and asymmetries (e.g. many small positive moves but few large negative ones) that we require us to use specialised metrics to assess their representativeness.

To recap: using deep generative models (GANs, VAEs, EBMs) for synthetic data generation is an extremely fruitful area of ML research. They are already proofs of concepts from research groups as well as some early industrial products; their usefulness will vary wildly among different applications both in terms of data types (images, speech, tabular, etc.) as well as application fields (healthcare, retail, etc.). Therefore we have no established way to generally quantify their impact on the performance of a model. That said, we will have a clearer view in a couple of years; GDPR/CCPA are powerful motivators for certain multi-billion dollar companies to convince legislators about the benignity of their data usage - synthetic data generation is a piece of that puzzle.

Answer 5

After long time, I would conclude the answer is no, based on some quite solid theoretical basis https://en.wikipedia.org/wiki/Data_processing_inequality

Answer 6

For visual tasks, data augmentation can often be accomplished by rotation, scaling, or rearranging patches. These transformations do not necessarily add information, but can be useful for models to learn to generalize better.

The generator in a GAN learns a complex distribution from its training data from which you can sample, and you can view these new samples as another type of transformation of the original data, akin to rotation, scaling, etc.

Answer 7

2023: debates goes on https://arxiv.org/abs/2305.17493v2

We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear.