I am working on a class project where I compare the performance of GAN and WGAN. Since the only difference between GAN and WGAN is the Wasserstein loss, I chose one neural network model architecture and trained both GAN and WGAN (so, only the loss functions differ).

However, WGAN performs much worse than GAN, and I'm not sure why. Is the performance of Wasserstein loss model dependent? If had to compare GAN and WGAN, holding the NN architecture fixed, what architecture should I choose?

  • $\begingroup$ WGAN requires that you also clip the model parameters for the critic, did you do so? Also note that a recent paper has shown a few limitations to this WGAN approach and has improved on it by using a gradient penalty (see this paper: arxiv.org/pdf/1704.00028.pdf ) Using such a WGAN-GP could help you get better results. $\endgroup$ – GR4 Mar 27 '18 at 8:30

Usually, the same architecture and parameters would not be good for training both GAN and WGAN.

In a typical GAN, you want to avoid making the discriminator more powerful than the generator, and you want to avoid training the discriminator so much that it "overpowers" the generator and always finds the fakes.

In WGAN, you want to make the discriminator as powerful as possible, possibly by giving it a larger network, and you also want to train it for as long as computationally feasible -- several iterations for every one iteration the generator trains. The theory behind WGAN requires that the discriminator has converged to the optimal discriminating function, so this is important.

If for some reason, you really need to fix one architecture, choose one where the generator is about the same size as the discriminator, and then make sure when you're training the WGAN that you really train the discriminator a lot -- maybe 10x more than the generator.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.