GAN : Why does a perfect discriminator mean no gradient for the generator? (Note : this is a cross-post from Artificial Intelligence. As I got no answer there in two weeks, I'm trying my luck on a more popuplated SE site. I know this is against SE's policy on cross-posting and that I should "post it on the most adequate site in the first place", but I feel that here is as much as an adequate place as over on AI. Anyway, feel free to close if you think it's the right thing to do given the SE/CV policy. Thanks).

In the training of a Generative Adversarial Networks (GAN) system, a perfect discriminator (D) is one which outputs 1 ("true image") for all images of the training dataset and 0 ("false image") for all images created by the generator (G).
I've read on separate occasions that -when trained with the original GAN framework and loss function as described in the Goodfellow 2014 paper- if this "perfect D state" occurs during training then this is a "failure mode", sometimes refered to as "vanishing gradients" or "convergence failure", from which G "cannot recover".
For instance :

"When the discriminator is perfect (...), the loss function falls to zero and we end up with no gradient to update the loss during learning iterations" (source, well-recieved blog post)


"An optimal discriminator doesn't provide enough information for the generator to make progress." (source, Google developers)


[T]he generator score approaches zero and does not recover. (...)
In this case, the discriminator classifies most of the images
correctly. In turn, the generator cannot produce any images that fool
the discriminator and thus fails to learn. (source, Mathworks)

The Towards Principled Methods for training GANs paper (Arjovsky & Bottou, 2017) is perhaps a more accurate source on the matter as it dives deeper in the theory by proving that since the loss function relates to the Jensen-Shannon distance, it saturates when the distributions of real and generated images are disjoint (hence no gradient).

If the two distributions we care about have supports that are disjoint
or lie on low dimensional manifolds, the optimal discriminator will be
perfect and its gradient will be zero almost everywhere. (section 2.2)

I understand the gist of the theory developed in that paper, but the result seems nevertheless counter-intuitive to me when I try to think about them in the context of the GAN training procedure. Indeed, my undertanding of a GAN training iteration is the following :

*

*Show true images to D, train D to output 1.

*Show false (created by G) images to D, train D to output 0.

*Show false images to D, train G such that D outputs 1.
I can see that in the case of a perfect D, steps 1 and 2 would lead to a loss of zero and hence no gradient.
But I would expect that in step 3 :

*

*As a perfect D would predict all images to be 0 and the loss would be computed by comparing its answers to predicting all 1s, we'd have a high loss.

*Thus, we would get a high gradient.

*By backprop, this gradient would lead to the identifcation of the most salient features in the images which D is using to predict them as false.

*This would provide valuable information to G to improve its false images to better match the training set.

So looking at it that way, it doesn't seem to me that a perfect D should lead to "vanishing gradients" and G being unable to recover from it.
What's wrong with my understanding of the training process and why it is not compatible with the results from the Arjovsky paper ?
 A: In the original formulation of Goodfellow et al. article, the Generator G is trained to minimize $$\mathbb{E}_{z \sim q(z)} [\log (1-D(G(z))]$$.
This is the formulation which is used to describe the issue in Arjovsky et al, 2017.
It is easy to see that if the discriminator is perfect, i.e. $D(G(z)) = 0 \; \forall z$, the loss is also 0.
A better formulation for the generator loss is the non-saturating version described in the 2016 tutorial, which has better convergence properties
$$-\mathbb{E}_{z \sim q(z)}[\log D(G(z))]$$
The authors state that this formulation is also present in the original GAN article, but I only found it in  (Eq. 13)  of the 2016 paper.
This is further emphasized in the article Many paths to equilibrium: GANs do not need to decrease a divergence at every step, where the statement of Arjovsky's paper is criticized

This figure is used by Arjovsky et al. (2017) to show that a model
they call the “traditional GAN” suffers from vanishing gradients in
the areas where D(x) is flat. This plot is correct if “traditional
GAN” is used to refer to the minimax GAN, but it does not apply to the
non-saturating GAN. (Right) A plot of both generator losses from the
original GAN paper, as a function of the generator output. Even when
the model distribution is highly separated from the data distribution,
non-saturating GANs are able to bring the model distribution closer to
the data distribution because the loss function has strong gradient
when the generator samples are far from the data samples, even when
the discriminator itself has nearly zero gradient
While it is true that the $\frac{1}{2}\log(1-D(x))$
loss has a vanishing gradient on the right half
of the plot, the original GAN paper instead recommends implementing $-\frac{1}{2}\log D(x)$. This latter,
recommended loss function has a vanishing gradient only on the left side of the plot. It makes sense
for the gradient to vanish on the left because generator samples in that area have already reached the
area where data samples lie.



As a perfect D would predict all images to be 0 and the loss would be
computed by comparing its answers to predicting all 1s, we'd have a
high loss.

In the formulations I provided the Generator learns only with the Discriminator 1s (i.e the generator classifies a generated image as real). So in this case the loss would be zero in both formulations.
