1
$\begingroup$

I have used the GANs (Generative Adversarial Networks) with a binary cross-entropy loss function (in both generator and discriminator). Throughout the training step, the variation of generator loss and discriminator loss for 400 epochs as following:

enter image description here

Does this variation look like correct?

What is the stop criteria of gans if we use the loss function values? Can I use early stopping?

$\endgroup$
2
$\begingroup$

The loss values obtained while training a GAN are almost never reliable. In most GAN papers, there are "qualitative" and "quantitative" evaluation methods that are used in judging the performance of a GAN. The qualitative methods are generally subjective, involving human observers who are tasked with evaluating whether a sample is real or fake. In such cases, the "goodness" of the generated sample matters i.e. if the generated sample looks good then the GAN training is successful, and so on (the loss values can fluctuate even if the generator produces realistic samples). However, qualitative evaluations are biased and do not paint a complete picture (the generated images look good even in the case of mode collapse but are extremely less diverse).

Therefore, using the loss values directly is not recommended in GANs. Instead, metrics such as the Inception Score, Frechet Inception Distance (FID score), and perceptual similarity measures (LPIPS) are used for interpreting the results. To answer your question here, the above quantitative metrics can be effectively used for early stopping i.e. stopping the training when FID score worsens or perceptual similarity isn't improving, etc. A comprehensive list of metrics used in GAN evaluation is provided in this paper.

As mentioned in the previous answer, there are other GANs models such as WGAN, WGAN-LP that provide a better mathematical insight into the GAN training by calculating the Wasserstein distance, and enforcing Lipschitz continuity.

$\endgroup$
1
$\begingroup$

The standard GAN loss isn't useful in assessing whether training has converged or finished. There are some GANs, such as WGAN, which minimize the Wasserstein distance between the distribution of real and generated images (in which case the loss is interpretable, and can be used to judge convergence). Finally, you can measure the Frechet Inception Distance (a standard metric for evaluating GANs) periodically to judge when training has converged.

$\endgroup$
6
  • $\begingroup$ thanks shimao, can I use the gans for the network data, or they are specifically used for the image? $\endgroup$ – Ramzi Aug 18 '20 at 14:22
  • $\begingroup$ you can use GANs for whatever you want $\endgroup$ – shimao Aug 18 '20 at 14:32
  • $\begingroup$ How could I use the generator loss to stop training the WGAN? $\endgroup$ – Ramzi Aug 20 '20 at 12:09
  • $\begingroup$ the generator loss is not interpretable in wgan $\endgroup$ – shimao Aug 20 '20 at 14:09
  • $\begingroup$ however the discriminator loss measures the negative wasserstein distance bt the generator distribution and the data distribution, and can be used to determine when training has converged $\endgroup$ – shimao Aug 20 '20 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.