1
$\begingroup$

If a GAN generator has the same (but reversed) hidden layer architecture as the discriminator, is a the discriminator's loss expected to be approximately double the generator's?

In the examples I'm learning from, the:

  • Discriminator loss is the sum of a batch of real and a batch fake images losses.
  • Generator loss is from only one batch of image losses

All batch sizes are the same.

Intuitively, there are twice as many examples in the discriminator loss, so I'd expect it to be twice as large.

Is there any reason not to scale the discriminator loss by dividing it by $2$?

Are batch sizes used in the loss functions usually all the same size?

$\endgroup$
1
$\begingroup$

The loss is usually averaged across the batch, so that should not be a factor. In any case, being off by merely a factor of 2 is unlikely to cause too many problems -- the real failure mode to watch out for while training GANs is mode collapse.

$\endgroup$
  • $\begingroup$ Yes, the loss is averaged over the batch, but there are two batches considered for the discriminator loss - the generated and the real examples. Assuming the batch sizes are the same, and a perfectly matched discriminator and generator, then wouldn't the summed loss of the two batches be twice the loss of a single batch? $\endgroup$ – Tom Hale Apr 16 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.