Initialization of GAN discriminator

Question

The question is pretty straightforward: how are GAN and WGAN discriminators typically initialized?

I couldn't find much info on this. E.x. for GANs, I imagine you would theoretically want the discriminator to not be very good initially, otherwise your generator cannot start learning. You also don't want the discriminator to be too bad, so maybe a good initial discriminator is the one which outputs 0.5 on average. However, do you want this discriminator to output 0.5 to all samples, or maybe 0 to one half and 1 to the other half? I suppose for WGANs this matters less in theory.

Answer 1

You can use almost any standard weight initialization schemes such as Glorot, He, or similar variants.

Typically, a good initialization scheme will result in outputs which are fairly small, and if your classification logits are fairly small, then the output probability will be fairly close to 0.5.

Intuitively, this is a pretty reasonable way to initialize, surely without seeing any data, you wouldn't want to be very confident of classifying any image. On the other hand a nearly uniform prior is a good starting point.