I am working on a class project where I compare the performance of GAN and WGAN. Since the only difference between GAN and WGAN is the Wasserstein loss, I chose one neural network model architecture and trained both GAN and WGAN (so, only the loss functions differ).
However, WGAN performs much worse than GAN, and I'm not sure why. Is the performance of Wasserstein loss model dependent? If had to compare GAN and WGAN, holding the NN architecture fixed, what architecture should I choose?
Usually, the same architecture and parameters would not be good for training both GAN and WGAN.
In a typical GAN, you want to avoid making the discriminator more powerful than the generator, and you want to avoid training the discriminator so much that it "overpowers" the generator and always finds the fakes.
In WGAN, you want to make the discriminator as powerful as possible, possibly by giving it a larger network, and you also want to train it for as long as computationally feasible -- several iterations for every one iteration the generator trains. The theory behind WGAN requires that the discriminator has converged to the optimal discriminating function, so this is important.
If for some reason, you really need to fix one architecture, choose one where the generator is about the same size as the discriminator, and then make sure when you're training the WGAN that you really train the discriminator a lot -- maybe 10x more than the generator.
Try to substitute gradient clipping with gradient penalty in WGAN, if you haven't done so yet. The important thing is that you should NOT use batch normalization in WGAN discriminator, as it breaks the whole idea. The authors of the WGAN paper suggest to use layer norm.
