Reading here, I found:

If the discriminator wins by too big a margin, then the generator can’t learn as the discriminator error is too small.

This is something I read somewhere else as well, but I can't really get it. If the discriminator has a low loss, it means that when I give it a fake sample (with "fake" label) it gives me a low score (assuming its output is "probability of real") with high certainty, so I can imagine that the gradient of the error will be small.

When I train the generator, I pass the same fake image, but with the "real" label. In this case, I expect that the gradient of the error should be high, since we are basically telling the discriminator that it's making a mistake (and a big one, if the discriminator loss was low), so the error gradient should be high, and this gradient will be the one going to the generator for training.


You might find the answer in this paper "Towards principled methods for training generative adversarial networks" (https://arxiv.org/pdf/1701.04862.pdf). It has a part explaining why the generator's gradient vanishes as the discriminator gets stronger.

  • 4
    $\begingroup$ Thanks! I'm accepting the answer, although I think you should actually explain it rather than just linking the paper. $\endgroup$
    – rand
    Apr 11 '18 at 13:33
  • $\begingroup$ In fact, I was looking for the answer myself and found this paper. I haven't finished reading it yet. lol. $\endgroup$
    – kangzheng
    Apr 13 '18 at 3:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.