3
$\begingroup$

It seems that, if the GAN generator is simply mapping noise to a value which should be as indistinguishable as possible for the discriminator from the real data, the generator could simply learn to map all noise to a single (or perhaps a handful) of real examples. For instance, in the image classification task, what stops a GAN generator from ignoring (through close-to-zero weights) the noise it receives and then mapping to a single image of a "9" every time?

$\endgroup$
1
$\begingroup$

For one, the Discriminator would quickly learn to categorize all 9's as fake samples, which would lead the Generator to try to generate different samples.

Nonetheless, overfitting on samples or modes in the training set is indeed a real problem for GANs. In the above example, when the Discriminator categorizes all 9's as fakes, the Generator might start generating only 8's and so on.

There have been several suggestions to improve the behavior of this problem. Some focus on improving the formulation of the loss function, eg the Wasserstein GAN distance, others have focused on methods that normalize across batches. A recent technique that appears to work well is spectral normalization.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.