5
$\begingroup$

(Note : this is a cross-post from Artificial Intelligence. As I got no answer there in two weeks, I'm trying my luck on a more popuplated SE site. I know this is against SE's policy on cross-posting and that I should "post it on the most adequate site in the first place", but I feel that here is as much as an adequate place as over on AI. Anyway, feel free to close if you think it's the right thing to do given the SE/CV policy. Thanks).


In the training of a Generative Adversarial Networks (GAN) system, a perfect discriminator (D) is one which outputs 1 ("true image") for all images of the training dataset and 0 ("false image") for all images created by the generator (G).

I've read on separate occasions that -when trained with the original GAN framework and loss function as described in the Goodfellow 2014 paper- if this "perfect D state" occurs during training then this is a "failure mode", sometimes refered to as "vanishing gradients" or "convergence failure", from which G "cannot recover".

For instance :

"When the discriminator is perfect (...), the loss function falls to zero and we end up with no gradient to update the loss during learning iterations" (source, well-recieved blog post)

"An optimal discriminator doesn't provide enough information for the generator to make progress." (source, Google developers)

[T]he generator score approaches zero and does not recover. (...) In this case, the discriminator classifies most of the images correctly. In turn, the generator cannot produce any images that fool the discriminator and thus fails to learn. (source, Mathworks)

The Towards Principled Methods for training GANs paper (Arjovsky & Bottou, 2017) is perhaps a more accurate source on the matter as it dives deeper in the theory by proving that since the loss function relates to the Jensen-Shannon distance, it saturates when the distributions of real and generated images are disjoint (hence no gradient).

If the two distributions we care about have supports that are disjoint or lie on low dimensional manifolds, the optimal discriminator will be perfect and its gradient will be zero almost everywhere. (section 2.2)

I understand the gist of the theory developed in that paper, but the result seems nevertheless counter-intuitive to me when I try to think about them in the context of the GAN training procedure. Indeed, my undertanding of a GAN training iteration is the following :

  1. Show true images to D, train D to output "True".
  2. Show false (created by G) images to D, train D to output "False".
  3. Show false images to D, train G such that D outputs "True".

I can see that in the case of a perfect D, steps 1 and 2 would lead to a loss of zero and hence no gradient.
But I would expect that in step 3 :

  1. As a perfect D would predict all images to be "False" and the loss would be computed by comparing its answers to predicting all "True"s, we'd have a high loss.
  2. Thus, we would get a high gradient.
  3. By backprop, this gradient would lead to the identifcation of the most salient features in the images which D is using to predict them as "false".
  4. This would provide valuable information to G to improve its false images to better match the training set.

So looking at it that way, it doesn't seem to me that a perfect D should lead to "vanishing gradients" and G being unable to recover from it.

What's wrong with my understanding of the training process and why it is not compatible with the results from the Arjovsky paper ?

$\endgroup$

1 Answer 1

2
$\begingroup$

In the original formulation of Goodfellow et al. article, the Generator G is trained to minimize $$\mathbb{E}_{z \sim q(z)} [\log (1-D(G(z))]$$.

This is the formulation which is used to describe the issue in Arjovsky et al, 2017. It is easy to see that if the discriminator is perfect, i.e. $D(G(z)) = 0 \; \forall z$, the loss is also 0.

A better formulation for the generator loss is the non-saturating version described in the 2016 tutorial, which has better convergence properties

$$-\mathbb{E}_{z \sim q(z)}[\log D(G(z))]$$

The authors state that this formulation is also present in the original GAN article, but I only found it in (Eq. 13) of the 2016 paper.

This is further emphasized in the article Many paths to equilibrium: GANs do not need to decrease a divergence at every step, where the statement of Arjovsky's paper is criticized

This figure is used by Arjovsky et al. (2017) to show that a model they call the “traditional GAN” suffers from vanishing gradients in the areas where D(x) is flat. This plot is correct if “traditional GAN” is used to refer to the minimax GAN, but it does not apply to the non-saturating GAN. (Right) A plot of both generator losses from the original GAN paper, as a function of the generator output. Even when the model distribution is highly separated from the data distribution, non-saturating GANs are able to bring the model distribution closer to the data distribution because the loss function has strong gradient when the generator samples are far from the data samples, even when the discriminator itself has nearly zero gradient While it is true that the $\frac{1}{2}\log(1-D(x))$ loss has a vanishing gradient on the right half of the plot, the original GAN paper instead recommends implementing $-\frac{1}{2}\log D(x)$. This latter, recommended loss function has a vanishing gradient only on the left side of the plot. It makes sense for the gradient to vanish on the left because generator samples in that area have already reached the area where data samples lie.

plot of generator losses in https://arxiv.org/pdf/1710.08446.pdf

As a perfect D would predict all images to be 0 and the loss would be computed by comparing its answers to predicting all 1s, we'd have a high loss.

In the formulations I provided the Generator learns only with the Discriminator 1s (i.e the generator classifies a generated image as real). So in this case the loss would be zero in both formulations.

$\endgroup$
7
  • $\begingroup$ Thanks for your answer ! I had forgotten about the non-saturating version of the objective... Two questions come to mind : $\endgroup$
    – Soltius
    Commented Feb 17, 2023 at 16:49
  • $\begingroup$ 1. Does the three-step training process I state in the question describe only this non-saturating version (and not the original version) ? I would think so, as Alg 1 in the original GAN paper does not seem to work with any "label flipping" (i.e. step 3 in the training process as I understand it), while the 2016 paper you linked does mention it when presenting the new objective : "Instead of flipping the sign on the discriminator’scost to obtain a cost for the generator, we flip the target used to construct the cross-entropy cost" $\endgroup$
    – Soltius
    Commented Feb 17, 2023 at 16:54
  • $\begingroup$ 2. Why hasn't this non-saturating rewriting solved most of the "no gradient" training problems that still seem to be here nowadays (cf quotes in my question) ? I gather that conventional wisdom is that training should be balanced between D and G, but if my understanding of loss values is correct when using this non-saturating version, then why should we care about that ? On the contrary, wouldn't it be better to start with a really good D, which would yield high losses and gradients for G (this does not seem to be the consensus eg this answer) ? $\endgroup$
    – Soltius
    Commented Feb 17, 2023 at 17:00
  • $\begingroup$ I don't know if I understood correctly point 1, but after a second read to the algorithm you proposed, and if at each step backpropagation is performed, giving the discriminator a batch of all-real or all-fake images would likely result in a bad training (my intuition is that catastrophic forgetting would happen at each step). I have yet to figure out if both algorithms work if we flip 1s with zeros. $\endgroup$
    – Ciodar
    Commented Feb 17, 2023 at 21:10
  • 1
    $\begingroup$ In all implementations that I came across, batches were composed of either all real or all fake images (makes it easier to code I guess), eg here (first google hit for "gan training"). I think alternating fake/real at each batch is a very high frequency from a network training pov, more than enough to prevent catastrophic forgetting (which could happen if eg we showed real images for 10 epochs than fake images for the 10 next ones, etc). $\endgroup$
    – Soltius
    Commented Feb 17, 2023 at 22:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.